Prompt-Based Methods and Instruction Design

Suppose you have a language model that is already trained and you want it to behave the way your application needs. It might need to follow a particular output style, answer questions in a specialized domain, or produce a fixed format every time. There are two broad ways to get there. You can leave the model alone and change what you send to it, or you can change the model itself. This article is about that choice, and about the vocabulary you need to reason about it.

The chapter this article belongs to asks how to adapt a language model to your needs. That question depends on history, because the tools you will meet later were shaped by which architectures dominated the field and what changed. The source names two historical facts that set the stage: an architecture called the LSTM once led deep learning for natural language processing, and a different architecture, the transformer, took over that role in 2018. Keep those two names and that date in mind, because the rest of the article builds on them.

LSTM, a type of recurrent neural network, was the dominant architecture of deep learning for NLP before the transformer architecture took over in 2018. The word dominant needs care. It does not mean LSTM was the only option or the best option in every case. It means that when researchers and practitioners building NLP systems reached for a deep learning architecture, LSTM was the default choice for a long period. Dominance of this kind is a fact about adoption across a field, which is why a change of dominance is a historical event and not a single technical breakthrough.

The source describes the change as a handoff. It does not describe a gradual blending in which the two architectures slowly shared the field. The order of the two eras is shown below.

Prompt-based methods adapt a model by giving it instructions, context, and tools. Prompting leaves the trained model as it is and shapes what you send to it. Finetuning instead changes the model's weights. The same underlying model can be steered either way, but only finetuning alters the numbers inside it.

The source says finetuning can be applied to the whole model or to part of it. The diagram below contrasts these two cases using a simplified model made of three layers. The layers are a teaching device, not a claim about any particular architecture.

Finetuning is useful because it can produce models that are more customized to your needs. The source is clear, however, that this comes with more up-front investment. Compared to prompt-based methods, finetuning incurs a much higher memory footprint. At the scale of today's foundation models, naive finetuning often requires more memory than what is available on a single GPU. For this reason the source describes finetuning as expensive and challenging to do.

Follow the state of the system as you move from prompting to finetuning. With a prompt-based method, the model's weights are not changed, so the work is confined to shaping the input you send. With finetuning, the weights are the thing being adjusted, so the training process has to hold and update that model. The source does not break down the exact memory categories involved in training, but the core difference is already visible: one approach changes the model's weights and the other does not.

The source says that a question it hears very often is when to finetune and when to use RAG, but it does not answer that question in this passage. It does set up the trade-off you need to reason about: finetuning is more customized and requires more up-front investment, while prompt-based methods have a much lower memory footprint. The scenario below is an original illustration of how that trade-off plays out.

Parameter-efficient finetuning (PEFT) is a memory-efficient approach to finetuning that has become dominant in the finetuning space. Its full name signals the goal: adapt the model while using fewer trainable parameters than full finetuning would require. The source says the chapter explores PEFT, how it differs from traditional finetuning, and an overview of its evolving techniques.

PEFT is a family of approaches rather than a single technique. Adapter-based techniques are one category within PEFT. The diagram below places them in that structure. The source does not list the other PEFT categories, so the branch for them is left as a general placeholder.

The distinction to hold onto is this. Traditional finetuning trains the whole model or a part of it, and it has the high memory footprint described earlier. PEFT is designed to reach a similar adaptation goal while keeping the number of trainable parameters small. Adapter-based techniques belong to the PEFT branch, not to traditional finetuning.