Prompt Engineering Versus Finetuning

Suppose you have a pretrained language model and you want it to behave differently on one particular job. One option is to retrain every parameter in the model. That works, but it is expensive. Parameter-efficient finetuning, usually shortened to PEFT, asks a narrower question: how little trainable machinery can we add and still steer the model?

This section surveys the main answers the field has settled into. Before reading on, it helps to make a prediction about the first of those answers. The word adapter is a hint, so think about what an adapter could plausibly touch.

PEFT methods in this section fall into two buckets. Adapter-based methods add modules or parameters to the model weights. Soft prompt-based methods add special trainable tokens to the input. Every technique discussed in this section belongs to one of these two branches, so the split is the first thing to hold onto.

The diagram shows the branches and the techniques named in the reading. It does not list every method that appears in the section's comparison table. The reading also says that many PEFT frameworks implement these techniques out of the box, so in practice you usually choose among them rather than building them from scratch.

The word adapter is a useful handle. An adapter-based method does not reach into the input at all. Instead, it changes the model's own structure by attaching extra parameters, and the adaptation lives in those attached parameters. The original model's parameters are the foundation, and the new parameters are where the task-specific change is stored.

The reading gives Houlsby et al. (2019) as an example of an adapter method that set this pattern. The source does not describe that method's internal design, so this article will not describe it either. What we can say is that the attached modules sit alongside the model weights, and the adaptation is stored there rather than in the input.

Soft prompt-based methods make their change on the input side rather than inside the model's parameters. The model still receives a sequence of tokens, but some of those tokens are special ones whose values were learned during tuning. They are called soft prompts because, like ordinary prompts, they guide the model's behavior. The difference lies in what they are made of.

The reading also describes soft prompting as a kind of crossover between prompt engineering and finetuning. A figure in the source shows that hard prompts and soft prompts can be combined to change a model's behaviors. So the two are not always rival choices. In some settings they work together.

The source separates soft prompts from hard prompts along two lines. Hard prompts are the ordinary text prompts a person writes. Soft prompts are the learned tokens described above. The two properties the source emphasizes are readability and form, and a third property, trainability, follows from them. The comparison below lays out how the two differ.

Read the columns row by row. Readability asks whether a person can look at the prompt and understand it, and hard prompts pass while soft prompts do not. Form asks what the prompt is made of: discrete words for hard prompts, continuous vectors for soft prompts. Trainability asks whether tuning can change the prompt, and that is the key distinction. Hard prompts are static, while soft prompts can be optimized through backpropagation.

Soft prompt tuning is a subfield with several techniques whose names sound alike, which can be confusing. The source lists prefix-tuning, P-Tuning, and prompt tuning. They differ mainly in where the soft prompts are inserted, and that location is the most useful thing to remember when telling them apart.

The diagram is a generated illustration of the source's statements, so read it as a map of insertion points rather than a claim about every implementation. Prompt tuning acts once, at the embedded input. Prefix-tuning acts at each layer. Adapter modules are attached to the model's weights, which is a different kind of change from adding tokens to the stream at all.

Which PEFT methods do practitioners actually use? The source answers with a proxy. Its author analyzed over 1,000 open issues on the GitHub repository huggingface/peft in October 2024. The assumption is that people who use a technique are more likely to report issues or ask questions about it. For P-Tuning, the search covered both 'p_tuning' and 'p tuning' to account for different spellings.

A bar chart in the source reports the number of issues for each technique, and its caption describes this count as a proxy for popularity. The chart is a plot of counts rather than a diagram with parts to label, so the useful content is the conclusion the source draws from it. The source states that LoRA dominates. It also says that soft prompts are less common, but that interest in them appears to be growing among people who want more customization than prompt engineering provides and who do not want to invest in finetuning.

The source uses this evidence to motivate the next section. Because LoRA is so widely used, the following section explains how LoRA works and how it addresses the challenge posed by early adapter-based methods. The source adds that the deep dive should give you a framework for exploring other finetuning methods even if you never use LoRA itself.