Prompting Techniques for Language Models

Suppose you want a language model to reason through a problem before it commits to an answer. You could simply ask it to show its work in the prompt. But that only helps if the model already knows how to show its work, and that skill has to come from somewhere. This article approaches chain-of-thought reasoning from the training data side. The question we will keep returning to is: what must the example responses in a finetuning dataset look like if a model is going to learn to think in steps?

The contrast becomes clear when two training examples sit side by side. The source describes two examples a dataset builder might write for a question about nitrogen. In the first, the instruction asks for the boiling point of nitrogen and the response is a single value, -320.4F. The response contains no reasoning at all. In the second, the instruction explicitly asks the model to reason step by step, and the response walks through an apple problem one stage at a time before arriving at the total. Both examples come from the same research source, and together they show the two response styles.

The word chain is doing real work. In a CoT response, each step is built on the step before it, so the final answer is supported by a sequence of smaller claims rather than appearing from nowhere. The apple response in the source follows this pattern: it states a fact, performs an operation on that fact, and then uses the result of that operation as the input to the next operation. A reader can check each link, and a model trained on such responses learns to produce intermediate results in the same order.

A model learns the patterns that appear in its training examples. If every example in a finetuning set jumps straight to a final answer, the model has little reason to produce intermediate steps when it faces a hard question. The source makes this point directly: to teach a model to generate step-by-step responses, its training data should include CoT responses. Prompting alone is not enough when the examples the model learned from never showed reasoning.

The evidence cited in the source supports this. In work by Chun et al. (2024), adding step-by-step responses to finetuning data improved performance on CoT tasks for models of several different sizes. For some tasks, accuracy nearly doubled. The improvement comes from the data itself, which is why the construction of CoT examples matters for anyone building a training set.

If CoT data helps so much, why not simply have plenty of it? The source identifies the bottleneck: generating multi-step responses is slow and laborious. Explaining how to solve a problem step by step is far more demanding than writing down only the final answer. A bare answer such as 9 takes seconds to record, while the apple response requires the writer to state each intermediate value correctly. Because of this extra effort, CoT datasets are less common than other instruction datasets.

For dataset construction, this scarcity means that each CoT example costs more to produce, so a builder cannot assume that a large pool of step-by-step responses already exists. Every example that shows intermediate steps has to be written with the same care that each link requires, which limits how quickly such a dataset can grow.