Model Hosting and Benchmarking

When people evaluate a model, they usually ask whether it performs well. That question matters, but a second question matters just as much: where did the model's knowledge come from, and what rules govern that origin? The answer to the second question is called data lineage.

This article follows one line of reasoning. We start with the origin of training data, follow it through the licenses and terms attached to it, and then look at how evaluation has to account for what the model has absorbed. By the end, you should be able to look at a model and ask the questions that lineage makes necessary.

Lineage is a history, not a single fact. A model is not simply trained on 'data.' Its training data can come from several places, and each place may have passed the data along through other parties. The lineage question is therefore a chain of questions: where did each input come from, and who set the rules for it at each handoff?

The source notes that models are likely trained on data scraped from the internet. It also describes a second kind of input: a model's training data can come from the outputs of another model. A model built this way does not stand alone. It sits at the end of a chain that began with the open web, with another model, or with both.

Read the diagram from left to right. Each input on the left arrives with its own history and its own conditions. The rules attached to upstream sources travel with the data, so they still matter after the data has been combined and used to train the model. This is why lineage must be traced rather than assumed.

The source also gives a public-interest argument for openness here. Because the public generated the content that models are trained on, some people argue that the public should have the right to access the training data. Whether or not you agree with that position, it shows why lineage is a subject of debate and not only a technical detail.

A model can be governed by more than one set of rules at once. Its own license states what its owner allows. But the model may also be built from material whose owner set separate restrictions. A license that permits something does not automatically override a restriction placed upstream in the chain.

The source offers an analogy from software licensing. The Elastic License forbids companies from offering the open source version of Elastic as a hosted service, and from competing with the Elasticsearch platform. The restriction concerns how the software can be used commercially. A model's training data can carry similar restrictions, which is why a permission on one layer is not enough on its own.

Lineage explains what a model may have absorbed. Model evaluation asks whether the model behaves correctly with that knowledge. One of the harder parts is knowledge boundaries. A model playing a role should respond the way the character would, and that includes not knowing things the character would not know.

The source uses a roleplaying example built around Jackie Chan. If Jackie Chan does not speak Vietnamese, then the roleplaying model should not speak Vietnamese either. Evaluating that model means deliberately testing for the boundary. You ask about the thing the character should not know and check whether the model reveals it.

The source calls this the negative knowledge check: a test of whether a model reveals things it should not know. It matters a great deal for games. An NPC that accidentally reveals story spoilers to players breaks the experience. So the check is not only about accuracy in general. It is about whether the model keeps to the limits that its role requires.

The comparison shows the two sides of the evaluation. The should-know side checks that the model answers within its role. The should-not-know side is where the negative knowledge check applies: for each item the role must not know, the evaluator asks about it and verifies that the model does not reveal it.

The reasoning patterns below come from the same logic the source uses. Each one tends to feel reasonable until the lineage is traced.

The source describes a lineage review as a fixed set of questions asked about each model before its evaluation results are trusted. The questions follow from the examples above, and you should work through them in order for any model you are asked to assess. Three threads run through them: the origin of the training data, the rules attached to each layer, and the knowledge boundaries the model must respect.