Synthetic Data for Tool Use

Suppose you want a model to find the cheapest train ticket between two cities. A person would typically open a browser, type the route into a search bar, and click through the results one by one. That routine works well for a person. The question this article opens with is what a model should be trained to do instead.

The rest of this article follows the path from a human's routine to a usable training example. You will trace how a demonstration changes, how a tool use exchange is structured as a sequence of messages, and how a training dataset is judged by three criteria.

A natural first idea for teaching a model to use tools is to watch people do the job and record what they do. If a person can search the web, book a room, or run a spreadsheet calculation, a faithful record of that person's steps seems like the best possible training material. The belief is not entirely wrong, but treating human behavior as the ideal target is the mistake to avoid.

The source makes a direct claim: human annotations are not automatically ideal for AI agents. The reason is that the efficiency of an action depends on who performs it. Human annotations are the written records that a person produces when showing how to finish a task. What is efficient for a human, such as clicking through a visual results page, may be wasteful or unnecessary for a model that can call an interface directly.

Adjusting a demonstration means changing its form while keeping the knowledge it carries. Take the ticket search again. The human routine contains a browser, a search bar, and a manual scan of results. The agent-ready version keeps the goal, which is finding the cheapest ticket on a route, but replaces the visual steps with a direct request to the search interface. The diagram below shows the substitution.

Notice what does not change. The goal stays the same, and the person's knowledge of which search matters is still the reason the example exists. Only the steps that were shaped by human convenience are rewritten for the model.

None of this means human expertise is useless. The source recommends using domain experts to write tool use data. In that setup, each prompt is a task that needs tools, and each response is the set of actions that completes it. Experts know what the tasks are and which tools they require. The caution is about the form of the demonstration, not the value of the knowledge behind it.

Experts have a second limitation. When asked to explain their work, they may leave steps out. They may forget a step, or judge it unimportant even when it changes the outcome. For this reason the source says it is often necessary to observe how humans perform these tasks in order to ensure accuracy. Observation catches what the explanation omits, so a good process combines expert knowledge of the task with direct watching of the work.

Before collecting data, the source asks you to decide how the interaction should be structured. Your application may need single-turn data, multi-turn data, or both. The distinction depends on how many exchanges a task requires before it is finished. A single-turn example contains one prompt and one answer. A multi-turn example contains several exchanges that lead toward a finished result.

Ordinary conversation data assumes a simple rhythm. The user speaks, the AI replies with one message, and the turns alternate. Tool use breaks that rhythm. The AI may need to produce several messages within a single turn, and each message may go to a different destination.

The source's example makes the problem concrete. An AI sends one message to a code interpreter, which runs the code, and another message to the user to explain what it is doing. Both messages come from the same AI turn, yet they travel to different places. A plain alternating transcript has no way to show that difference.

To handle this, the Llama 3 authors designed a multi-message chat format. It has two main parts. Message headers specify the source and destination of each message. Special termination tokens mark where the human and AI turns begin and end. Together these let training data express which message goes where. A training pipeline that stores each turn as one undivided message would lose the destination information the model needs to learn.

Read the diagram from top to bottom. The user produces only the first message. The AI produces two messages, but they are not sent to the same place. Only the message headers tell the model which message is code for the interpreter and which is an explanation for the user.

Curation is the work of deciding what data a model learns from. The source says data curation follows three criteria: data quality, data coverage, and data quantity. Each application may need data with different characteristics, and different training phases may need different data mixes, but these three criteria apply throughout.

The source offers a cooking analogy. Think of model training as cooking, where the data fed into the model are the ingredients. Data quality is the quality of the ingredients, since a meal cannot be good if the ingredients are spoiled. Data coverage is the right mix of ingredients, such as not using too much or too little sugar. Data quantity is how many ingredients you should have.

The three criteria are separate questions, so a dataset can succeed on one and fail on another. A large collection of spoiled examples has quantity but not quality. A small set that contains only one kind of interaction may have quality but poor coverage. A good decision checks all three rather than relying on the one that is easiest to measure.

Curation is often pictured as adding material, but the source defines it in two directions. Data curation means creating new data to help a model learn new behaviors and removing existing data to help a model unlearn bad behaviors. Removal matters as much as creation.

Notice how the three criteria apply to this fix. The unsolicited-rewrite examples are a data quality problem, because they teach a bad behavior. The new fact-checking examples are about data coverage, because they supply the missing kind of interaction. Removing the bad examples and adding good ones are both curation decisions.