Synthetic Data Generation

Building the data for a model can feel like a vast, messy task. Yet the reasoning behind it is simpler than it first appears. Synthetic data generation is one way to produce that data, and it is practical for many use cases. But generating data is only the first step, and the way you judge that data matters as much as how you made it. This article follows one dataset through a short sequence of checks, so you can see how the three core criteria of dataset design work in practice.

A useful way to begin is to ask what you want the model to be able to do. Once you have that answer, you build a collection of examples that demonstrates exactly that behavior. In other words, the dataset is designed backward from the behavior you want, not forward from whatever files happen to be available.

This reversal changes how you think about the work. A common shortcut is to gather whatever material is at hand and hope it contains what the model needs. Designing backward starts from the target behavior and asks what examples would show it. Dataset creation is therefore both a technical question about what the model should learn and an organizational question about who is responsible for gathering and checking the material. The source notes that teams now create dedicated data roles whose job is to acquire suitable datasets while protecting privacy and meeting compliance requirements.

Dataset design rests on three core criteria: quality, coverage, and quantity. These criteria apply across training phases. Each one answers a different question about the same collection of examples. The diagram below shows how the three criteria relate to one another and to the dataset as a whole.

The three criteria are not independent checkboxes. High-quality data with sufficient coverage matters as much as how much data you have. A small amount of high-quality data can outperform a large amount of noisy data, so quantity alone is not enough. Treating quantity as the only headline measure leads teams to overlook quality and coverage.

The data you need depends on more than your application. It also depends on the training phase. Three phases are named: pre-training, instruction finetuning, and preferred finetuning. Pre-training requires different data from instruction finetuning, and both differ from preferred finetuning. The contents change from one phase to the next.

The lens you use to judge the data does not change. Each phase is still evaluated through quality, coverage, and quantity. So when you move from one phase to another, you ask the same three questions about different material. The answers to those questions will shift, but the questions stay the same.

High-quality data is hard to acquire, so many teams turn to synthetic data, meaning data produced programmatically. Generating data this way has long been a goal. It became a practical solution for many more use cases once AI could create realistic, complex data. The chapter this article belongs to focuses in particular on synthesizing instruction data for finetuning.

Generation comes first, but it does not feed training directly. The generated output must pass an evaluation step before it is used to train a model. The flowchart below shows where that evaluation sits in the sequence.

Evaluating AI-generated data is just as tricky as evaluating other AI outputs. People are more likely to use generated data when they can reliably evaluate it. A generation method is only as useful as your ability to judge its output. If you cannot check the data, you have little basis for trusting it.

The following mistakes follow directly from the criteria and the evaluation gate described above.