Instruction Finetuning Data Formats

Instruction finetuning teaches a model to follow directions by showing it many examples. Each example pairs a request with an answer. Building a large collection of these pairs by hand is slow, so practitioners often ask an AI model to help produce them. The useful question is therefore not whether AI should be involved at all, but which parts of each pair it should produce.

A single training example always has two parts, so there are several ways to assign the work. AI can be used to synthesize the instructions, the responses, or both. A team could let AI draft the requests while people write the answers. The reverse arrangement is also possible, with people writing the requests and AI producing the answers. Both halves can also be generated by AI.

Read the diagram column by column. Each column is one possible division of labor for the same two-part record. The choice among them is a design decision about where human judgment is most needed, not a rule that one arrangement is always correct. The source material describes all of these options as valid ways to assign the work.

The main risk when generating instructions is coverage. If an AI model is simply asked for instructions with no structure, the results may cluster around a few themes and leave large parts of the intended use case unrepresented. The remedy is to impose structure before generating anything. Coverage then becomes a matter of counting: every topic on the list receives its share of instructions.

A common starting point is a list of topics, keywords, or instruction types that the dataset should contain. For each item on that list, you generate a chosen number of instructions. The topic list does not have to be written by a person. Both the topic list and the templates can be generated by AI. A practitioner can ask a model for a broad set of subjects, review them, and then use that reviewed list as the skeleton for instruction generation.

Once instructions exist, each one needs a response. You can generate one or more responses per instruction. Producing more than one response for the same instruction gives the dataset alternative answers to the same request. This follows from the one-to-many arrangement used for response generation: one instruction can lead to several responses.

UltraChat is a multi-turn dialogue dataset that follows the topic-first pattern. Its authors first asked ChatGPT to generate 30 topics about various aspects of daily life, including technology, food and drink, fashion, nature, education, finance, and travel. For each topic, they asked the model to produce 30 to 50 subtopics. The same model was then used to generate instructions and corresponding responses for those subtopics.

Trace the flow step by step. The process moves from broad topics, to narrower subtopics, to the instruction-response pairs themselves. Each stage narrows the focus, so that the final pairs are specific enough to cover the range of daily-life subjects that the topic list set out.

Alpaca shows the other pattern, starting from examples rather than from topics. Stanford researchers began with 175 instruction-response examples taken from the Self-Instruct seed dataset. Those seed examples were written to cover a diverse and interesting range of uses. The Alpaca authors then used a GPT-3 model, text-davinci-003, to generate 52,000 instruction-response pairs that mirrored the seed examples.

Read the pipeline from left to right. The seed set is small and carefully written. The generation step takes its pattern from those seeds, and the output is a far larger set. The dataset grows from 175 seeds to 52,000 generated pairs, and each generated pair follows the pattern of a seed rather than being a copy of it.