Seed Examples and Self-Instruct

Instruction finetuning teaches a model to follow directions by showing it many examples. Each example pairs a request with an answer. Building a large collection of these pairs entirely by hand is slow, so practitioners often ask an AI model to help produce them. The useful question is therefore not whether AI should be involved at all, but which parts of each pair it should produce.

A single training example always has two parts: an instruction, which is the request, and a response, which is the answer. Because there are two parts, there are several ways to assign the work between people and AI. AI can be used to synthesize the instructions, the responses, or both. A team could let AI draft the requests while people write the answers. The reverse arrangement is also possible, with people writing the requests and AI producing the answers. Both halves can also be generated by AI.

Read the diagram column by column. In the first arrangement, the model drafts the requests and a person supplies the answers. In the second, the roles are reversed. Neither arrangement is inherently correct; the source frames the choice as a question of which half each party should produce for a given dataset.

The main risk when generating instructions is coverage. If an AI model is simply asked for instructions with no structure, the results may cluster around a few themes. Large parts of the intended use case can then go unrepresented, even though the dataset looks large. The remedy is to impose structure before generating anything, so that coverage is planned rather than hoped for.

A common starting point is a list of topics, keywords, or instruction types that the dataset should contain. Templates can play the same role. For each item on that list, you generate a chosen number of instructions. Coverage then becomes a matter of counting: every topic receives its share. Once instructions exist, each one needs a response, and you can generate one or more responses per instruction. Generating more than one response gives the dataset alternative answers to the same request.

The topic list does not have to be written by a person. Both the topic list and the templates can be generated by AI. A practitioner can ask a model for a broad set of subjects, review them, and then use that reviewed list as the skeleton for instruction generation. Human review sits between the AI-generated list and the generation step that depends on it.

Two real datasets show the two patterns. UltraChat is a multi-turn dialogue dataset that started from topics. Its authors first asked ChatGPT to generate 30 topics about various aspects of daily life, including technology, food and drink, fashion, nature, education, finance, and travel. For each topic, they asked the model to produce 30 to 50 subtopics. The same model then generated instructions and corresponding responses for those subtopics. The pipeline moved from broad topics, to narrower subtopics, to the instruction-response pairs themselves.

Alpaca shows the other pattern, which starts from examples rather than topics. Stanford researchers began with 175 instruction-response examples taken from the Self-Instruct seed dataset. Those seed examples were written to cover a diverse and interesting range of uses. The Alpaca authors then used a GPT-3 model, text-davinci-003, to generate 52,000 instruction-response pairs that mirrored the seed examples. Each generated example follows the pattern of a seed example, so the seed set sets the style and the range that the generated set inherits.

Notice the scale change. The starting set holds 175 items, and the output holds 52,000. The source does not describe the intermediate counts, so the lesson to take from it is the direction of the process: a small, carefully chosen set of seed items steers the generation of a much larger set.