Prompt Engineering for Model Comparison

Imagine a small support team that wants a model to draft replies to customer tickets in its own house style. The team has a few hundred good example replies, a modest budget, and one shared GPU. Before any training begins, the team has to make decisions, and the source pack frames the mechanics of finetuning as reducing to three of them: a base model to start from, a finetuning method that decides which parts of the model get updated, and a framework or service that actually runs the training.

The source is clear that finetuning projects involve harder questions before these three choices appear. You have to decide whether finetuning is worth the effort at all, gather training data, and keep the resulting model up to date. Once those questions are settled, the finetuning step itself is comparatively simple. The rest of this article follows the three choices in turn, and then shows how they interact, because each choice affects the others.

Choosing a base model uses the same criteria introduced earlier in the course for model selection, and those criteria apply to both prompt-based work and finetuning. The source names three: model size, licenses, and benchmark performance. Size affects the cost and the hardware needed to run training and serving. The license determines what you are legally allowed to do with the model. Benchmark performance gives a rough sense of how capable the model is on tasks that others have already measured.

The source offers a starting strategy for the early part of a project, when you are still checking whether the task is feasible at all. Begin with the most powerful model you can afford. If that model cannot produce good results, a weaker model is unlikely to do better, which suggests the task or the data probably needs work first. If the strongest model does meet your needs, try weaker and cheaper models and measure each one against the strong model you started with. The strongest model therefore becomes the benchmark for the rest of the search.

By the time a team reaches finetuning, it has usually run prompt engineering experiments and has a sense of how different models behave. The source describes two development paths for organizing the work from there, both drawn from OpenAI's finetuning best practices. They differ in where the work begins and what the final model is for.

The progression path validates a finetuning setup in sequence: first the code, then the data, then the experiments. The distillation path works the other way around in terms of its starting resources. It begins with a small dataset and a strong model, and the strong model produces the data used to train a cheaper model. Because the two paths answer different questions, they are not competing answers to the same problem. A team can use both, for example by distilling a cheap model after a progression run has shown that the training setup works.

Once a base model is chosen, the next decision is how much of the model to update. Full finetuning updates all of the model's parameters. Adapter-based techniques update only a subset of the parameters, and LoRA is the example the source names. The two methods make different demands on data and on serving.

Serving cost is the second criterion. With LoRA you serve a single full model, while full finetuning means serving multiple full models. The support team's few-hundred-example dataset and single shared GPU point toward an adapter-based method, which matches the source's advice to start with something like LoRA and attempt full finetuning later.

The framework decision answers a practical question: who runs the training, and on what hardware. The source describes an easy route and a flexible route, and a third option for training across more than one machine. The diagram below sets out the order in which to check project constraints.

The easiest route is a finetuning API. You upload data, select a base model, and get back a finetuned model. APIs can come from model providers, cloud service providers, or third-party providers. Their limits are that you can only use the base models the API supports, and the API may not expose every setting that affects performance. The source's verdict is that APIs suit someone who wants something quick and easy, and they can frustrate someone who wants more control.

The flexible route is to finetune with an open-source framework. The source names LLaMA-Factory, unsloth, PEFT, Axolotl, and LitGPT, and notes that these support a wide range of methods, especially adapter-based ones. For full finetuning, many base models publish their open-source training code on GitHub, which you can clone and run with your own data. The trade-off is that you must provision the compute yourself. The source estimates that a mid-tier GPU may be enough if you only do adapter-based work, and that a framework integrated with your cloud provider can help when you need more compute.

A single machine is not always enough. To finetune a model across more than one machine, you need a framework built for distributed training. The source lists DeepSpeed, PyTorch Distributed, and ColossalAI as examples.

The three decisions are sequential in the sense that each one narrows the next, but they also influence each other. The source gives the examples that a larger base model may need more compute, a method that updates fewer parameters may fit a small dataset better, and a managed API may be the quickest route while limiting which models are available. The diagram below places the three decisions as parts of one finetuning project.