Concepts / Creating a Runbook for a Software Project

Creating a Runbook for a Software Project

A showcase README has six sections in a fixed order: project description, impact statement, architecture diagram and explanation, runbook, eval set, and demo video.

  • Programming

Why the First Lines Decide the Reading

A showcase README is the page a reader sees when they open a portfolio project. Often that reader is a hiring manager who has a few minutes to decide whether the project deserves a closer look. This article follows the path a showcase README takes from its first sentence to its last section, and it shows why that path is ordered the way it is. The plan is to test a common belief about how to open such a page, then learn the six sections in their fixed order, and finally look at how the proof-bearing sections give the reader grounds for trust.

What do you think happens?

A showcase README begins with a plain-language description of the project. What do you think the second section is, the one that tells the reader what the project changes in the world?

  • The architecture diagram and explanation

  • The impact statement

  • The runbook

  • The demo video

Reveal answer
Answer:

The impact statement

The second section describes the impact. The source expects it to include metrics or statistics and to be honest and realistic. Architecture comes third, after the reader already knows what the project does and how well it worked.

The Belief That Technical Detail Impresses

Many people building a portfolio project assume that the README should prove how advanced the work is. The reasoning feels sensible: the most technical terms are placed up front, so the reader sees the framework names, the model family, and the pipeline architecture immediately. The belief says that a reader who is impressed by the vocabulary will be impressed by the work.

The source points in the opposite direction. The people who read these READMEs are often hiring managers, and they need to understand what the project does before they can judge how well it was built. A reader who cannot tell what the project is for has no context in which the technical terms mean anything. Technical vocabulary placed before purpose tends to force the reader to guess at relevance, and guessing is where interest is lost.

Framework and modelnamedtechnical terms firstCustomer-facingpurposeplain language firstReader cannot tellpurposeno context for termsReader judgesrelevanceknows what it doesReader moves oninterest lostTechnical choices inarchitectureread with contextthenthenthenthen
How does a reader's attention move through the page when the README opens with technical detail, compared with when it opens with a plain-language description of the project?

The correction to the misconception is not to hide the technology. The source says to put the customer-facing purpose first and let the technical choices appear later, in the architecture section, where they have context. The technology is still in the README. Its position changes, and that position determines whether the reader can make sense of it.

Rewriting an Opening Sentence

The first sentence of the README describes what the project does in plain language, in the customer's terms. The source contrasts two ways of saying the same thing. Both describe the same project, but only one tells a non-specialist what happens and who it helps.

VersionSentenceWhat a non-specialist learns
Technical versionAn LLM-based audio pipeline built on a named frameworkA technology stack, but not what it does or for whom
Customer-facing versionA voice agent that handles routine freight dispatch calls for a regional brokerWhat happens during the call, and which business it serves

Two opening sentences that describe the same project, taken from the source

The source ties this directly to reader behavior. The plain-language sentence is what separates a project a hiring manager spends about three minutes on from one they close. Notice what the customer-facing version does: it names an activity (handling routine dispatch calls), a setting (a regional broker), and a kind of work (voice). A reader can picture the project from that alone. The technical version asks the reader to already know what an LLM-based audio pipeline is before it says anything useful.

The Six Sections in Their Fixed Order

The source states that a showcase README has six sections, and that they follow a fixed order. Each section answers a different question for the reader, and the order matters because each section builds on what the previous one established. The reader first learns what the project is, then how well it worked, then how it is built, then how to run it, then how its results are checked, and finally sees it in motion.

Project descriptionwhat the project doesImpact statementwhat it changesArchitecture diagramand explanationhow it is builtRunbookhow to run itEval sethow results are checkedDemo videothe system in motionthenthenthenthenthen
Where does each of the six sections sit in the README, and which section comes immediately after the impact statement?

Read the sequence as a path of trust. A reader who stops early still walks away with the most important facts: what the project is and what it changed. Later sections deepen the picture for a reader who keeps going. The architecture section comes third, after the impact statement, which means the reader already has a reason to care about the design. The impact statement comes immediately after the project description and before the architecture section.

  • 1. Project description: a plain-language opening sentence about what the project does, in the customer's terms
  • 2. Impact statement: what the project changed, with metrics or statistics that are honest and realistic
  • 3. Architecture diagram and explanation: one image and one paragraph describing how the project is built
  • 4. Runbook: how to clone the project, install its dependencies, run the demo, run the eval suite, and produce the dashboard
  • 5. Eval set: the evals folder and its README, documenting the rubric, data source, labeling protocol, and version history
  • 6. Demo video: a screen-recorded, narrated walkthrough showing the system handling a real input and producing a real output

Writing an Honest Impact Statement

The second section describes the impact. The source expects it to include metrics or statistics, and it must be honest. Padding numbers or overstating results damages trust, and a reader who notices an overstatement will doubt the rest of the page. The goal is a claim that is specific about the measure, the scope, and the duration, and that does not claim more than was actually covered.

Worked example

An impact statement from a real pilot

Write an impact statement for a voice agent that handles routine freight dispatch calls, using the results of a six-week pilot with one customer.

  1. 1
    Name the measure

    The source uses average call handling time. The claim is about that measure, not about overall business value.

  2. 2
    Give the before and after values

    The source reports a drop from 4 minutes to 38 seconds for the broker's most repetitive call class.

  3. 3
    State the scope

    The claim applies to the most repetitive call class, not to all calls the broker handles.

  4. 4
    State the duration and the number of customers

    The result comes from a six-week pilot with one customer. Saying so keeps the claim within what was measured.

Result

Reduced average call handling time from 4 minutes to 38 seconds for the broker's most repetitive call class, over a six-week pilot with one customer. The claim is specific about the measure, the scope, and the duration, and it does not claim more than the pilot covered.

The source also covers a case where no production deployment exists. The honest alternative it gives is a sentence of this kind: ran against 500 sampled calls from a public dataset, correctly handled 87 percent of them per the rubric in the evals folder. The result is smaller than a production figure, but every part of it can be checked, and the rubric is named. A smaller claim that the reader can verify is worth more than a larger claim they cannot.

The Architecture Paragraph

The third section is the architecture diagram and one paragraph of explanation. The source is specific about its size: one image and one paragraph. The paragraph must name five things: the components, the data flow, the model, the eval suite, and the deployment surface. The source sets the reading target at about two minutes, which is enough for a hiring manager to form a sensible next question.

Because the paragraph has to name all five elements, it works as a checklist. If any of the five is missing, the reader has to guess how the parts connect. For example, a paragraph that names the components and the model but never mentions the eval suite leaves the reader unsure whether the system was measured at all. Keep the paragraph to one block of prose so it stays short enough to read in the time the source allows.

Runbook, Eval Set, and Demo Video

The last three sections are the ones that let a reader verify the project for themselves. Each one gives a different kind of proof, so each is explained separately below.

How the Proof Sections Build Trust

The runbook, eval set, and demo video each connect to the reader's trust in a different way. The runbook lets the reader run the project. The demo video shows the project running on a real input and shows the author presenting it. The eval set documents how results were checked. The diagram below shows which of these supports which kind of trust.

Runbookclone, install, runProject runsreader can try itReader trustin the project's claimsEval setrubric and data sourceWork was evaluatedresults can be checkedDemo videoreal input and outputAuthor built itauthor presents the worklets reader run itshows it runningshows author presentingdocuments the rubricsupportssupportssupports
How does each of the runbook, eval set, and demo video connect to the reader's trust in the project's claims, and which of them supports which?

The source is direct about the eval set: without it, the project is only a demo. The runbook and video show that the project works, but the eval set is what shows the work was evaluated. A reader needs both kinds of evidence to trust a claim such as the 87 percent figure.

Common Mistakes in Showcase READMEs

  • Common mistake

    Opening with the framework and model names

    What it looks like

    An LLM-based audio pipeline built on a named framework

    Why it fails

    The reader learns the technology but not what the project does or who it helps, so the reader cannot judge relevance.

    Do this instead

    Open with the customer-facing purpose, such as a voice agent that handles routine freight dispatch calls for a regional broker, and move the technical names to the architecture section.

  • Common mistake

    Padding the impact statement with numbers that have no stated scope

    Why it fails

    A figure without a measure, a data source, or a duration cannot be checked, and a reader who spots overstatement will doubt the rest of the page.

    Do this instead

    Name the measure, the scope, and the duration, and claim no more than was covered, as the pilot example does.

  • Common mistake

    Leaving out the runbook

    Why it fails

    Without it, a hiring manager has no way to download and try the project, and the source notes that most candidates skip this section.

  • Common mistake

    Writing an architecture paragraph that omits one of the five required elements

    Why it fails

    The reader is then forced to guess how the components, data flow, model, eval suite, and deployment surface connect.

  • Common mistake

    Putting the demo video at the top of the README instead of the sixth position, which breaks the path of trust the order is meant to build.

Checking an Opening Line

EASY

Rewrite the following technical opening sentence so that a non-specialist can tell what the project does and who it helps: An LLM-based audio pipeline built on a named framework. Then write one sentence explaining what a reader learns from your version that they did not learn from the original.

Hints
  • Start with what happens during the work, not with the tools used to do it.
  • Name the people or business the project serves, using the customer's terms.
  • Compare your sentence with the freight dispatch example in the opening-sentence section before you finish.

Putting the Structure to Work

  1. 1
    The opening sentence describes the project in plain language and in the customer's terms, and the technical choices belong later, in the architecture section.
  2. 2
    A showcase README has six sections in a fixed order: project description, impact statement, architecture diagram and explanation, runbook, eval set, and demo video.
  3. 3
    The impact statement must be honest: name the measure, the scope, and the duration, and claim no more than was measured.
  4. 4
    The runbook, eval set, and demo video each give the reader a different way to verify that the project works and was evaluated.

Key Takeaways

  • The opening sentence describes the project in plain language and in the customer's terms, and technical choices appear later, in the architecture section.
  • A showcase README has six sections in a fixed order: project description, impact statement, architecture diagram and explanation, runbook, eval set, and demo video.
  • The impact statement must be honest, naming the measure, scope, and duration of any result, and it must not claim more than was measured.
  • The runbook, eval set, and demo video each give the reader a different way to verify that the project works and was evaluated.