Evaluating Model Performance on Downstream Tasks

Suppose someone hands you a model that is already trained, and your job is to make it perform better on a task. Where would you look first? Most people reach for the model: a different architecture, more parameters, or a new training recipe. That instinct is reasonable, but it is only one of two places an improvement can come from. The other place is the collection of examples the model learned from. This article follows that second path. It asks what happens when the model is held still and the dataset is the thing being changed, and why researchers and practitioners have started to treat that as a serious way to measure progress.

A benchmark is a fixed setup that lets people compare their work. The difference between the two families comes down to which piece is held constant. In the older model-centric style, the dataset is fixed and people compete to build the best possible model on it. ImageNet is the classic example of a dataset used this way.

In the data-centric style the roles are reversed. The model is fixed, and people compete to build the dataset that gives that particular model the best performance. The same model serves as the reference point for everyone, which means that differences in results can be attributed to the data each participant produced rather than to differences in model design.

The two styles are not opposites that cancel each other out. They are two different ways of asking where the leverage is. A model-centric benchmark asks how much better a model can get when the data is given. A data-centric benchmark asks how much better the data can make a model that is already given.

In 2021, Andrew Ng launched a data-centric AI competition. Every participant started from the same base dataset and was asked to improve it. The source names three concrete techniques: fixing incorrect labels, adding examples of edge cases, and augmenting data. Each of these changes what the model sees during training without touching the model itself.

Each technique targets a different weakness. A wrong label teaches the wrong lesson. A missing edge case leaves the model guessing on rare inputs. Augmentation expands the set of examples the model can learn from. A participant in such a competition is therefore doing data work: inspecting the examples, deciding which ones are faulty or missing, and changing the dataset accordingly.

A competition needs a judging rule that does not depend on opinion. DataComp, hosted in 2023, set a goal of creating the best dataset for training a CLIP model. The judging was mechanical. A standardized script trains a CLIP model on each submitted dataset, and the quality of that dataset is measured by how well the resulting model performs on 38 downstream tasks.

The key idea is that the dataset is never graded by reading it. It is graded by what it produces. In 2024, the same organizers ran a similar competition aimed at language models, with model scales from 412 million to 7 billion parameters. Other data-centric benchmarks exist as well, including DataPerf and dcbench. Together they show the same pattern: a fixed training procedure, a dataset as the submitted artifact, and a score based on downstream performance.

The source makes a strong claim about why data curation matters. The right data can make a model more capable, safer, and able to handle longer contexts. The reverse is also true: poor data can cause a model to increase its biases and its hallucinations. Mistakes in data do not only reduce accuracy in a narrow sense. They can harm the model and waste the resources spent training it.

Data curation is therefore described as a science that depends on understanding how the model learns and what resources are available to help it learn. Dataset building is also not a separate task that happens far away from model work. The source recommends that dataset builders work closely with application developers and model developers. On a small team one person may do all of it, while organizations with high data demands tend to create specialized roles. In both cases, what data is needed depends on the task and on what the model is supposed to learn.

Different training methods expect data in different shapes, so the format of a dataset is tied to the goal of training. For self-supervised finetuning, the data takes the form of sequences. For instruction finetuning, each example is an (instruction, response) pair. For preference finetuning, each example is an (instruction, winning response, losing response) triple, which gives the training signal a direct comparison to learn from.

A reward model can be trained using the same preference format, or using data where each example is an (instruction, response) pair with an annotated score, written as ((instruction, response), score). The format is not a formatting detail. A preference dataset with only single responses cannot teach the comparison that preference finetuning needs, and a sequence dataset cannot directly supply the instruction and response pairing that instruction finetuning relies on.

The source also notes that acquiring high-quality data annotations is always challenging, and that the difficulty grows when the goal is to teach complex behaviors such as chain-of-thought reasoning and tool use. The harder the behavior, the more the data has to demonstrate it, and the more carefully the annotations have to be made.