Selecting the Right Model for Your Application
Hard attributes are properties that are impossible or impractical for you to change, usually because of decisions by model providers (licenses, training data, model size) or your own policies (privacy, control).
A Candidate That Keeps Failing
Suppose you are choosing a model for a task, and a candidate performs poorly on your first few tries. It is tempting to start tuning right away. Before you decide whether to keep working on it, though, you need an answer to a more basic question: is the problem something you are able to fix, or is it a fixed property of the model that you have to accept?
Every property of a model falls into one of two groups. Some properties are off the table because you cannot change them, or because changing them would be impractical. Others are properties you can push on, and you may be willing to invest effort in doing so. Learning to tell these two groups apart is the first step in judging whether a model is worth the time you would spend improving it. The rest of this article builds that skill.
The Question to Ask First
When you meet a property of a candidate model, you can sort it by asking a short sequence of questions. The first question is whether the property traces back to a decision someone already made, either by a model provider or by your own policy. If it does, the property is hard. If it does not, you ask whether effort on your part can move it. If effort can move it, the property is soft. If effort cannot move it, or moving it would be impractical, the property is hard in practice.
Run the latency property through this sequence to see the method in action. Latency is how long a model takes to respond. Whether it is hard or soft depends on your relationship to the model, so the answer to the first question changes with the situation. The next section works through that dependence in detail.
Where Hard Attributes Come From
Hard attributes are properties that are impossible or impractical for you to change, usually because of decisions by model providers (licenses, training data, model size) or your own policies (privacy, control).
Hard attributes usually trace back to a decision that someone already made. Some of those decisions belong to the model providers: the license the model is released under, the data it was trained on, and its size. Other hard attributes come from your own policies, such as privacy requirements or how much control you need over the system.
Neither group is something you can edit by tuning the model for an afternoon. A license is a legal term, training data was chosen before you arrived, and a model's size is part of what it is. This is why hard attributes should be checked first. They can sharply narrow the candidate models before you have spent any effort on them.
What Effort Can and Cannot Move
What separates soft attributes from hard ones is that effort can move them. Soft attributes, such as accuracy, toxicity, and factual consistency, can be improved with effort, for example by restructuring a task into steps. The source describes this kind of improvement as changes to how the work is done, not changes to what the model fundamentally is.
The two categories are easiest to hold in mind as opposites. A hard attribute is fixed by a decision already made, and you can only accept it or choose a different model. A soft attribute is one you can push on, and the effort you spend is what determines how far it moves. Use the diagram as a reference when you sort a new attribute, but remember that the sorting itself depends on the model and the use case.
Accuracy Before and After Decomposition
The source gives a concrete case of a soft attribute moving. A model's accuracy stayed around 20% on the first few prompts. The author then decomposed the task into two steps, and accuracy rose to 70%. Watch what changed and what did not.
The model did not change at all. The way the task was posed did. This is the defining feature of a soft attribute: the lever is your method, not the model's identity. Accuracy here is a soft attribute because restructuring the work moved it.
Hard or Soft Depends on Your Access
The most important point to absorb is that hard and soft is not a fixed label attached to a property. It is a judgment about a specific model in a specific situation. The source makes this concrete with latency, which is how long a model takes to respond.
Classifying latency in two deployments
The same latency property appears in two setups. In setup A, you have access to the model and can optimize it. In setup B, the model is hosted by someone else. Classify latency in each setup and justify your answer.
- 1Identify the property
Both setups involve the same property: how long the model takes to respond. The property itself has not changed, so the classification must depend on something else.
- 2Ask whether effort can move it in setup A
You have access to the model, so you can optimize it to run faster. Effort moves latency, which makes it soft in setup A.
- 3Ask whether effort can move it in setup B
The model is hosted by someone else, so you cannot change how it runs. Effort on your side cannot move latency, which makes it hard in setup B.
- 4Name what changed
Your access to the model is what changed the classification. The property was identical in both cases.
Latency is soft in setup A and hard in setup B. The same property can be classified differently depending on your relationship to the model.
What do you think happens?
Your team runs an open model on its own servers and can tune how it runs. Is latency hard or soft for this model?
Reveal answer
Soft
Because you have access to the model, you can optimize it to run faster, so effort can move latency.
Overestimating How Far Attributes Move
Calling an attribute soft does not promise that it will improve enough to be useful. Estimating how much you can improve a soft attribute is tricky, because you must balance optimism against realism. The source reports both a success and a failure, and both are about soft attributes.
| Belief | What the source reports | Lesson |
|---|---|---|
| Any soft attribute will reach a useful level with enough effort | A model stayed unusable for the task even after weeks of tweaking, and the author gave up on it | Soft means it can move, not that it will move far enough |
| A structural change always produces a large gain | Splitting the task raised accuracy from about 20 percent to about 70 percent in one case | A gain is real but depends on the task and the attribute |
| Once an attribute is soft, the candidate is worth improving | Effort may be spent on a model that never reaches the level you need | Set a stopping point before you start investing |
Common errors when estimating how far a soft attribute can improve
Common mistake Treating a soft label as a promise of improvement
What it looks likeAssuming accuracy will reach the target after a few more rounds of tuning.
Why it failsCalling an attribute soft tells you it can move, not how far it will move.
Do this insteadEstimate a realistic range and decide on a stopping point before you begin.
Common mistake Assuming the model itself must change
What it looks likeTrying to retrain or swap models when the task was simply posed in one large step.
Why it failsThe source's accuracy gain came from restructuring the task while the model stayed the same.
Do this insteadTry changing how the work is done before concluding the attribute is fixed.
Common mistake Classifying a property once and never revisiting it
What it looks likeLabeling latency as hard without checking whether you now have access to the model.
Why it failsHard and soft depend on your relationship to the model and the use case, so the label can change.
Do this insteadReclassify the property whenever the model, the use case, or your access changes.
Classify Your Own Attributes
For each attribute below, decide whether it is hard or soft for the model described, and justify your answer. (a) The license a model is released under, for a model you plan to download and use. (b) Factual consistency of a model's answers, for a model you can prompt and restructure. (c) Latency of a model hosted by a vendor you do not control.
Hints
- Start with the first question: does a provider decision or your own policy fix the property?
- If it is not fixed by a decision, ask whether effort on your side can move it.
- For latency, the deciding factor is whether you can change how the model runs.
- Name the decision or the lever in each justification, not just the label.
Key Takeaways
- A hard attribute is a property that is impossible or impractical for you to change, usually because of decisions by model providers (licenses, training data, model size) or your own policies (privacy, control).
- A soft attribute, such as accuracy, toxicity, or factual consistency, can be improved with effort, often by changing how the task is structured rather than changing the model.
- Hard and soft is a judgment about a specific model in a specific situation. Latency is soft when you can optimize the model and hard when someone else hosts it.
- Check hard attributes first, because they can sharply narrow the candidate models. Pursue soft attributes with realistic expectations and a stopping point.