Synthetic Data for Language Models

Think about what makes a good classroom teacher. A student learns less from a teacher who explains things carelessly or answers questions with errors, and more from one who models clear, careful, thoughtful reasoning. Training a conversational AI works in a similar way. The people who write example conversations for the model act as its teachers, so the quality of those examples shapes how well the model learns to hold an intelligent conversation.

This article focuses on one stage of the training pipeline: the stage where people write demonstration data for a model that has already been pre-trained. You will see what makes this work hard, what it costs, who tends to do it, and what alternatives teams have explored when high-quality human writing is too expensive or slow to scale.

Many traditional data labeling jobs can be done with little or no specialized knowledge. Deciding whether a photo contains a bicycle is a task most people can perform after a short explanation. Demonstration data is different. A single prompt may require the labeler to reason through a problem, gather relevant information, and decide whether the user's request is appropriate to fulfill at all.

Because each response is a worked piece of reasoning rather than a single tag, the labeler's skill becomes part of the training signal. The pipeline above is why demonstration data cannot be produced by just anyone: every step is a place where a weaker writer can leave a mark that the model will later reproduce.

The easiest way to see the stakes is to follow a single prompt through two different writing processes. The prompt is the same in both cases; only the response differs, and the model cannot tell the difference between them. It only sees a pair to imitate.

The before-and-after comparison in the source shows why this work is expensive. A task that takes seconds per item can be spread across huge numbers of cheap annotations. Writing a demonstration pair can take up to 30 minutes, so it must be budgeted very differently. A dataset of thousands of pairs quickly becomes a major expense.

To estimate scale, multiply the number of pairs by the time per pair to get labeling hours, and multiply the number of pairs by the direct cost per pair to get the direct cost. The table below applies those two multiplications to 30 minutes per pair and 10 dollars per pair, using the figures named in the source. Each row is a worked calculation for a different dataset size.

Because the work demands judgment and domain sense, companies tend to rely on highly educated labelers. The profile of the labeler pool matters for two reasons. It affects the quality of the writing, and it affects whose perspective ends up embedded in the model.

Labeler demographics matter because a skewed pool can carry bias into the model. The source gives one concrete case: a volunteer survey in which 90 percent of respondents identified as male. A pool like that can shape which perspectives the written examples reflect, and the model learns to imitate those perspectives along with the wording.

Since expert writing is slow and costly, not every organization can follow the high-quality human annotation route. Several alternatives have appeared, and each trades something away. Human expert writing, volunteer crowds, heuristic filtering of internet dialogues, and AI-generated data can all be compared on who produces the data, what it costs, and how reliable it is.

The volunteer example shows the trade-off clearly. LAION mobilized 13,500 volunteers worldwide, and together they produced a large body of conversations and ratings. The scale is impressive, but the pool was not carefully controlled for bias, so scale alone does not settle the question of whose perspective the data carries.

The heuristic example shows a different trade-off. DeepMind did not write new conversations. Its team filtered internet text for dialogues arranged in an alternating speaker format. A filter like this is cheap to run at scale, but it can only check the form of a conversation. It cannot confirm that the reasoning or judgment inside the conversation is strong.