Deploying AI Systems in Customer Environments

An AI system can look impressive in a controlled setting and still disappoint once it is placed in front of a real customer. The model that handled the curated examples cleanly may struggle with messy inputs, unfamiliar data, and the operational constraints of the customer's own systems. This difference is what this article calls the gap between lab and production. The source treats the gap as central rather than incidental: it describes the gap as the bottleneck that determines whether the entire AI economy delivers on its promise.

In the lab, the system is tested under conditions its builders can control. In production, it meets the customer's real data, real workflows, and real expectations. A system that is strong in the first setting can be weak in the second, and the gap is the distance between those two results. The diagram below shows the shape of this change: the same system that handled curated examples cleanly produces a weaker result once messy inputs and the customer's operational constraints are present.

Because the gap lives in the customer's environment, it cannot be closed from a distance. The source's insistence that engineers sit next to customers follows from this. Closing the gap means observing how the system behaves where it is used and adjusting it until its results there match what it achieved in the lab.

It helps to trace the path a system takes from a lab demo to stable production use and to notice where it can stall. The source describes closing the gap as engineers working alongside customers, shipping code, testing against eval sets, and measuring outcomes. Read in order, those activities form a path, and the diagram below places each one along it.

A common assumption holds that the gap between a promising prototype and a useful deployment is a sales or marketing problem. If customers are not getting value, the reasoning goes, the answer is a better pitch, a more polished demo, more licenses sold, or more salespeople in the field. The problem sounds like one of persuasion and reach. The source tests this belief and rejects it: sales activity, demos, and licenses do not close the gap on their own.

The source describes the substance of the FDE role as having stayed the same since the role's beginnings. Titles, compensation, and tools may change, but the underlying purpose of closing the gap has remained consistent. This is why the source describes the role's core responsibilities as enduring rather than tied to any particular technology. As defined in this course, the FDE's primary responsibility is to close the gap between lab and production performance by working inside the customer's environment.

Placed in relation to the three parties, the FDE works in the customer environment where the gap appears. The lab is where the system was built and first tested, and the sales team is where the system is presented and sold. The FDE's ownership sits in the space between them: taking a system that performed in the lab and making its results in the customer's environment match what it achieved there.

The source names a set of practices it calls the motion that closes the gap. It lists four: embedded delivery, eval as specification, code as artifact, and outcome as proof. The source says this motion has remained the same even as the surrounding tools and titles change. The diagram below shows how the four practices relate to the central goal, and each practice is introduced in its own subsection.