Model Ensembling Methods

A single finetuned model is often expected to handle more than one job. A team may want the same model to answer customer questions, summarize documents, and classify support tickets. Each of these is a task, and each task asks the model to learn something specific. The question this article answers is how to bring several tasks into one model, and what goes wrong along the way.

The source material describes three main routes. The first is finetuning on all tasks at once. The second is finetuning on tasks one after another. The third is finetuning on tasks separately and then merging the results. Each route trades cost, data requirements, and the risk of losing earlier learning in a different way. The sections below follow those three routes in turn, then compare merging with a related idea called ensembling.

The method is straightforward to describe but demanding in practice. Learning several skills at the same time is generally harder than learning one, so the approach typically needs more data and more training than a single-task finetune. The cost lies less in the procedure itself and more in assembling a dataset large enough to cover every task well.

Simultaneous finetuning is the only one of the three routes that sees all tasks in the same training run. That property makes it attractive when the data for every task is available at the start. It is also the reason its cost grows as more tasks are added to the mix, because every new task enlarges the dataset and the training needed to cover all of them together.

Sequential finetuning trains tasks one after another. The appeal rests on an assumption: that it is easier for a model to learn one task at a time than to learn all of them together. Each stage gets a focused dataset, and the team does not need to assemble every task's data before training can begin.

The assumption has a flaw. Neural networks are prone to catastrophic forgetting, a tendency documented in the research literature. A model can lose the ability to perform an old task once it has been trained on a new one, and the result can be a sharp drop in performance on earlier tasks. Sequential finetuning therefore gives you a clean training schedule, but it does not protect what the model learned first.

To see the trouble concretely, follow one model through a two-stage sequence. The example below is an illustrative case constructed for teaching; it is not drawn from the source. Suppose the model is first finetuned on a task that classifies movie reviews as positive or negative. At that point it handles the review task well.

This is the core of catastrophic forgetting. The problem is not that the second task is hard or that the model runs out of room. The update made for task B is applied to the same parameters that task A relied on, and it changes them without regard for what they were doing before.

Model merging takes a different route. The key difference from the first two routes is that no task is trained on top of another task's finetuned weights in a chain. Each task gets its own finetuning run, starting from the same model. Because each run focuses on one task, the model can learn that task better than it would when sharing its capacity with others. Afterward, the separate models are combined by mixing their parameters together.

Because the merge happens after finetuning rather than during it, the sequential forgetting problem does not arise. No task's training run ever overwrites another's, since the runs never see each other's updates. The output is a single model that performs multiple tasks.

The source gives a practical motive for merging. It is appealing when models must be deployed to devices such as phones, laptops, cars, smartwatches, and warehouse robots. On-device deployment is often difficult because the device has limited memory, and keeping a separate model for each task on the device quickly exhausts that space.

Merging addresses this directly. Rather than squeezing several models onto a device, the separate models are combined into one model that handles multiple tasks while requiring much less memory. Shipping one merged model instead of several finetuned models means one set of parameters to store and load.

The source names two further reasons that deployment on the device matters. Some data cannot leave the device, often for privacy reasons. Some devices also have limited or unreliable internet access. On-device deployment can additionally lower inference costs, since computation offloaded to user devices does not have to be paid for in data centers.

Model merging can also serve as a way to carry out federated learning, in which multiple devices train the same model on separate data. Suppose model X is deployed to several devices. Each copy of X keeps learning from the data on its own device. After a while there are several copies of X, each trained on different data. Those copies can then be merged into one new base model that contains the learning of all the constituent models. No single device had to share its raw data; each contributed through its own trained copy.

The idea of combining models to get better performance predates merging. The source traces it to model ensemble methods, and it quotes a definition of ensembling as combining multiple learning algorithms to obtain better predictive performance than could be obtained from any of the constituent learning algorithms alone.

The two ideas differ in where they combine models. Merging typically mixes the parameters of the constituent models together, so the result is a single model. Ensembling typically combines only the outputs, and it keeps each constituent model intact. In an ensemble, given a query, several models each produce an answer, and a final answer is then produced from those answers, for example by a simple majority vote or by another trainable machine learning module.

The source notes a trade-off. Ensembling can generally improve performance, but it has a higher inference cost because each request requires several inference calls. Merging avoids that per-request cost because the result is one model. The source also observes that many models near the top of the Hugging Face Open LLM Leaderboard are merged models, in the same way that model ensembles once dominated leaderboards.