Delivering AI Solutions: working with the customer's internal team

Picture a team that spends months building an AI system for a customer. The system is finally running, and the builders leave. The people who built it are no longer around, but the system still has to keep working. Someone on the customer's side has to notice when something goes wrong, check whether the system is still producing good results, and respond when it is not.

This is the situation that designing for handoff is meant to prepare for. Designing for handoff means shaping the project from its earliest days so that the customer's own team can run the system after the builders step away. The source is clear that this is not a final chore done at the end. It is a set of decisions that shape the project from its very first days.

Before handoff, the people who built the system are the ones operating it. They are close to its behavior and can respond directly when something changes. The customer's internal team is not yet the party responsible for keeping the system running. After handoff, that responsibility moves. The customer's team must be able to operate the system without the builders in the room.

The diagram below shows the shift in ownership. Notice that the roles do not simply swap. The change is in who holds the responsibility, and that is why the work done during the build has to prepare the customer's team in advance.

Instrumentation is the set of signals a system emits about its own behavior. The source states that instrumentation begins at the first commit, because telemetry is what makes a launch possible. It is not something added after the system is already live.

Laid out in order, these checkpoints show that handoff preparation runs through the whole project. The signals gathered from the first commit are the ones that let the team evaluate the system and adjust it later. The source also points forward to a loop that runs from telemetry back into evaluations and prompt changes, which Chapter 12 covers in depth. For this article, the key link is the direction of that dependence: early signals make later evaluation possible.

A customer dashboard is usually imagined as a place to read numbers. The source asks us to think of it differently. Its main job is to build confidence. When the customer can watch what the system is doing in real time, their worry about the unknown decreases.

That confidence matters for the engineer as well. The source notes that a live, visible dashboard supports the engineer's ability to make scope decisions while the work is underway, because the customer is more willing to let the engineer make those decisions when they can see what is happening.

Designing for handoff produces four specific artifacts for the customer's internal team: a runbook, a set of alerts, an eval set, and a playbook. The source describes them as part of the delivery. They are not extra documents written in a rush at the end. Each one serves a different purpose, and together they let the customer's team notice problems, check quality, and respond.

The diagram places all four artifacts under the delivered system, because each one exists to support the running system and is handed over with it. The sublabels summarize each role as the source describes it: alerts help notice problems, the eval set helps check quality, and the runbook and playbook help the team act. The source does not give step-by-step contents for each artifact, so the sections below describe their purpose rather than a fixed template.

Notice that every step in this trace depends on something built earlier. The alert needs early telemetry, the eval set needs quality checks defined during the build, and the runbook and playbook need to exist before the builders leave. This is the practical meaning of treating the artifacts as part of the delivery.