Evaluating AI Systems in Production

Much of the earlier material in this module covers how a team decides what to build and how it checks that the work is sound. This article turns to the point where that work reaches real users. The delivery phase is the stage where a developed system is deployed to a live environment and begins handling real interactions. Many projects that looked solid during development stumble at this point, so it helps to look closely at one delivery that went well and ask what made it work.

Delivery is not simply a switch that gets flipped at the end of development. The source material frames delivery as something that can be done well, and it links the quality of delivery directly to the quality of the work that came before it. In plain terms, the delivery phase can go well when scoping and validation have been done well. Scoping decides what the system is responsible for. Validation checks that the work is sound. Delivery is where the results of both are either confirmed or exposed.

The case used in this article is a customer-facing AI agent deployed for a large retail company. It is a useful teaching example because the outcome was clear: the agent went live within a short timeline, and the team was able to widen its role over time. The stages below follow the case in order. Read each stage and notice what is different about the system or its scope at that point.

The first stage shows the source's point about scope. In the case, the timeline from kickoff to a live customer-facing agent was four weeks. That figure matters less as a target than as evidence of what a bounded scope allows. The stages after the constrained launch are where the case shows its central pattern. The team did not wait for the system to be perfect before going live, and it did not release the system without limits. It moved forward in small steps, and each step was tied to what the evidence showed.

A team that has defined a narrow, testable responsibility for its system can move from kickoff to a working deployment quickly. A team with an open-ended scope usually cannot. The reason is that a narrow responsibility can be validated. The team knows what the system should do, can check whether it does it, and can tell when it has drifted. An open-ended responsibility gives validation nothing firm to test against, so the work of deciding whether the system is ready keeps growing.

The agent in the case did not go live with full freedom. This is the central idea of incremental deployment. A system is put into production with limits on what it can do, who it serves, or how much it acts on its own, and those limits are loosened over time. The constraint is not a sign that the system is untrusted in a permanent sense. It is a way of gathering evidence from real traffic while keeping human oversight close enough that problems can be caught and corrected immediately.

The diagram shows the defining feature of this approach. Constraints were relaxed incrementally, and the trigger for each relaxation was evidence. The evaluation results came in clean from the production traffic, and those results justified giving the agent more room. The decision to loosen a limit was tied to observed behavior, not to a calendar date or a feeling that the system was probably ready. Monitoring every interaction is expensive, which is exactly why it is a temporary condition and not a steady state. Each expansion sends the team back to the evaluated state, where the next decision depends on the next set of results.

The customer saw a live, working system early. The source material links that early visibility to trust, and it notes that this trust allowed the team to expand its scope quickly. This is the momentum effect. A delivery that goes well makes the next step easier to approve, which in turn makes the next delivery easier to plan. The summary of the case states it simply: delivery, done well, builds its own momentum.

The phrase momentum is worth taking literally. Momentum here is not enthusiasm. It is the accumulated evidence and confidence that reduce the friction of each later decision about scope, monitoring, and autonomy. The loop only works when each expansion is validated. Trust that is granted without evidence does not carry the same weight in the next decision.

The case is easy to admire and easy to misread. The mistakes below come up when teams try to copy the pattern without its underlying conditions.