Project Scoping

Build the evaluation set at the start of the engagement, before writing production code, standing up the integration, or deploying the model.

Programming

The Pull Toward Visible Progress

Most people want to begin a project by opening an editor and writing the first function. That urge is understandable, because code is visible progress. A project can have a clear name and still be too vague to act on. Project scoping is the process of defining a project tightly enough that you can start working on it. This article looks at the project brief, a practical tool for that job, and at how writing the brief becomes part of the work itself.

When a new project begins, there is a strong pull toward writing production code, wiring up the integration, and getting a model deployed so that something tangible exists. That instinct feels productive, but it skips a question that matters more than any of those tasks: how far is the current system from what the customer actually needs? Until that question has a measured answer, everyone involved is working from impressions rather than evidence.

What do you think happens?

At the start of a new engagement, which should come first: writing the production code, or measuring how the current system performs against what the customer needs?

Write the production code first, then measure it once it is ready
Measure the current system first, using an evaluation set and a baseline

Reveal answer

Answer:

Measure the current system first, using an evaluation set and a baseline.

The source material argues that measurement comes first because it is the tool that tells you what building work is actually required. Building before measuring means the work is sized by assumptions rather than evidence.

Why the Evaluation Set Comes First

The common misconception is that building the system is the first job, and that measuring it can wait until the system is ready. The source material argues the reverse. The corrected sequence starts with scoping and then moves directly to validation. During the first week of validation, the team builds an evaluation set, which is a collection of evaluations used to judge how the system performs.

Only after the evaluation set exists does the team run baseline performance against it. A baseline measures how the current state of the system performs on those evaluations before any production work is undertaken. The order matters: the evaluation set is the measuring instrument, and the baseline is the first reading taken with it.

In what order do the evaluation set, production code, integration, and deployment happen, and which step must come first?

During this same stage, the team deliberately holds back on three activities: writing production code, standing up the integration, and deploying the model. These are not forbidden forever. They are postponed until there is a measured baseline to guide them, so that the work they represent is sized against real evidence rather than assumptions. In the diagram, the three postponed activities are grouped into one step because the source treats them together.

Reading the Gap Between Baseline and Threshold

A baseline result on its own does not tell anyone whether a project is nearly done or barely started. Its meaning comes from comparison with the threshold the customer is paying for. The threshold is the level of performance the customer expects the finished system to reach. The baseline is where the system stands today. The distance between the two is the gap, and the gap is what the engagement is really about.

Term	What it represents	Where it comes from
Baseline	Where the system stands today	Measured by running baseline performance against the eval set
Threshold	The level of performance the customer expects the finished system to reach	The customer's expectation, which the customer is paying for
Gap	The distance between the baseline and the threshold	The difference between the two, and the main subject of the engagement

The three quantities that together show how far the project must travel

The baseline is presented to the customer next to the threshold, so both parties look at the same two numbers. This shared view is what turns a vague hope into a concrete discussion about effort. Without it, the customer may hold one picture of how close the system is and the team may hold another.

Once the baseline and threshold are side by side, the size of the gap becomes the main planning signal. The gap is used to adjust scope and timeline early. Doing this early avoids misaligned expectations and late scope creep, which are the problems that appear when the size of the remaining work is discovered only after building has already begun.

How does the size of the gap between the baseline result and the customer's threshold change the scope, the timeline, and what gets discussed with the customer?

Scoping a Hypothetical Engagement

Worked example

Using the gap to shape the first conversation

A customer wants a system that reaches a stated threshold of performance. Before anyone writes production code, the team builds an evaluation set and runs baseline performance. The baseline sits well below the threshold. How should the team use this result?

1
Build the evaluation set first
The team assembles the collection of evaluations that will judge the system. This happens in the first week of validation, before any production code is written.
2
Run the baseline against it
The baseline records how the current system performs on those evaluations. This is the starting point, not a verdict on the finished product.
3
Place the baseline beside the threshold
Both numbers go in front of the customer together. The distance between them is the gap, and it is now visible to both parties rather than held as an impression by one side.
4
Let the gap set scope and timeline
A large distance points toward a larger scope and a longer timeline to discuss. A smaller distance points toward a smaller scope. The team adjusts both early, so the expectation agreed with the customer matches the evidence.
5
Postpone the build activities
Production code, integration, and deployment wait until the baseline guides them. The work they represent is then sized against the measured gap.

Result

The team enters the build with a shared, measured picture of the gap. Scope and timeline were discussed against that gap, so late scope creep is less likely.

The Five Sections of the Project Brief

The project brief is built around five sections: the customer, the proposed system, the eval set, the first slice, and the success metric. The source presents these as a structure you should adopt as your own template, so you do not need to invent a new format each time. Each section answers a different question about the project, and together they turn a loose idea into something you can check.

What are the five sections of the brief, and how do they relate to the single project brief they make up?

Read the five sections as a set of questions rather than a form to fill out. The section names follow the source. The distinctions between them matter, and two of them are easy to confuse.

Customer: who the project is for.
Proposed system: what you intend to build.
Eval set: the collection of evaluations that will check the system.
First slice: the first piece of the system you will build.
Success metric: how you will judge the result.

The Plain-Language Test

The source gives a practical check for whether scoping is finished. Try to fill in all five sections in plain language, without jargon. If you can do that, the project is scoped well enough to begin. If you cannot, the project is not yet scoped tightly enough to start.

1
Write one plain sentence for the customer section that names who the customer is.
2
Write one plain sentence for the proposed system that says what it does, without technical terms.
3
Write one plain sentence for the eval set that says what the system will be checked against.
4
Write one plain sentence for the first slice that says what you will build first.
5
Write one plain sentence for the success metric that says how the result will be judged.
6
If any sentence needs a technical term to make sense, or you cannot write it at all, the project is not yet scoped tightly enough to start.

EASY

Take a project you are considering and try to write the five sections in plain language with no jargon. Note which section you struggled with most. That section is where the scope is still undecided.

Hints

Start with the customer. If you cannot name who the project is for in one plain sentence, return to that first.
A first slice that you cannot describe in one sentence is usually too large to start.

Your attempt (not saved)

Mistakes When Scoping a Project

Common mistake
Writing production code before measuring the current system
Why it fails
Building before measuring means the work is sized by impressions rather than evidence, and the actual gap to the threshold stays unknown.
Do this instead
Build the evaluation set and run baseline performance first, then postpone production code, integration, and deployment until the baseline can guide them.
Common mistake
Showing the customer only the finished-system goal and not the current baseline
Why it fails
Without the baseline beside the threshold, the two sides do not share one view of the gap, so expectations drift apart.
Do this instead
Present the baseline next to the threshold so both parties look at the same two numbers.
Common mistake
Ignoring the gap when setting scope and timeline
Why it fails
A scope and timeline set without reference to the gap are likely to be revised late, which produces scope creep and misaligned expectations.
Do this instead
Use the size of the gap to adjust scope and timeline early, in the conversation with the customer.
Common mistake
Treating a clear project name as proof that the project is scoped tightly enough to start
Common mistake
Confusing the eval set with the success metric
Why it fails
The eval set describes what the system is checked against, and the success metric describes how the result is judged. Merging them hides how the result will actually be decided.
Do this instead
Write the eval set and the success metric as separate sections of the brief, each with its own plain-language sentence.

The Brief as Part of the Deliverable

The source makes a claim that many learners find surprising: the brief is not just preparation for the project, it is part of the project's deliverable. If you can fill in all five sections in plain language, the brief is itself the first portfolio artifact.

1
Fill in all five sections of the brief in plain language, using the plain-language test.
2
Commit the brief to the repository README before writing a single line of code.
3
Revise the README as the project teaches you what you missed.

The source also says that the discipline of writing the brief is part of the work. Stating the scope in plain words is not overhead to be skipped. Writing the five sections is where many gaps are found, and finding them early is cheaper than discovering them halfway through the build.

Key Takeaways

Build the evaluation set at the start of the engagement, then run baseline performance against it, before writing production code, standing up the integration, or deploying the model.
The baseline gains its meaning from comparison with the customer's threshold. The distance between them is the gap, and the size of that gap is the main signal for shaping scope and timeline.
Show the baseline beside the threshold to the customer so both sides share one view of the gap, which avoids misaligned expectations and late scope creep.
The project brief has five sections: customer, proposed system, eval set, first slice, and success metric. If you cannot fill them in plainly, the project is not yet scoped tightly enough to start.
The brief is part of the deliverable. Commit it to the repository README before writing code, and revise it as the project teaches you what you missed.