Concepts / Delivering an AI System to Users

Delivering an AI System to Users

An FDE engagement is organized into three phases in order: early scoping, validation, and delivery.

  • Programming

A Common Picture That Hides Validation

Forward Deployed Engineering, or FDE, is the way this course describes building AI systems alongside a customer. The work is not a single build step. An FDE engagement is organized over time, and this article introduces the frame that most of that organization depends on. The frame has three phases, taken in order: early scoping, validation, and delivery.

Many people picture an AI project as a line with two parts. First the team builds the system, then it checks whether the system works. In that picture, testing is just the tail end of building. The check happens after delivery, it is often framed as a review, and its results are sometimes filed away without action. Teams that hold this picture tend to treat validation as a sub-phase of delivery, or as a post-launch review. The rest of this article shows why that picture causes trouble, and what the three-phase frame does differently.

What do you think happens?

If a team has already finished building a system and then runs a review that nobody acts on, which phase is that review standing in for, and what would a working engagement do differently?

Reveal answer
Answer:

The review is standing in for validation, which has been moved to the end of the work. A working engagement would have tested the feasibility and potential impact of the impact area before building, so that the decision about whether delivery is worth attempting is made before construction begins.

The source frames validation as the phase that tests feasibility and potential impact and decides whether delivery should be attempted at all. Placing that test after delivery leaves the decision with nothing left to change.

Three Phases in a Fixed Order

An FDE engagement, in this frame, runs in a fixed order. Scoping comes first, validation second, and delivery last. Each phase has a distinct purpose, and each one feeds the next. The order is not arbitrary. The later phases depend on decisions made in the earlier ones, so the work of building a deliverable does not begin until the impact area has been chosen and tested for feasibility and potential impact.

Each phase answers a different question. Scoping asks where the work should focus. Validation asks whether that focus is worth building around. Delivery asks how to put a working result into the hands of the people who will use it. Keeping these three questions in mind makes it easier to tell which phase a piece of work belongs to.

Early scopingWhere the biggest impactcan beValidationIs this focus worthbuilding aroundDeliveryWorking result in users'handschosen impact areadecision to pursue impact area
In what order do scoping, validation, and delivery happen, and what carries work from one phase into the next?

The diagram shows the order and the hand-off between phases. Scoping produces an understanding of the problem space, and validation takes the impact area that scoping identified and tests it. Delivery starts only after that test, so the team enters delivery already holding a decision that the impact area is worth pursuing.

Scoping: Finding Where Impact Lies

Scoping is the initial phase. Its job is to understand where an AI system can have the biggest impact. According to the source, scoping is done by a small team working closely with the customer, so the phase is collaborative from the start. The customer's problems and context shape the search for an impact area from the first day.

Definition

Scoping: the initial FDE phase in which a small team, working closely with the customer, identifies where an AI system can have the biggest impact. Its output is an understanding of the problem space, not a finished system.

Validation Tests the Chosen Focus

Validation is the phase where the feasibility and potential impact of the project are tested. The source describes it as choosing the impact area before committing to the build. This is the discipline of deciding where to focus before spending effort on construction. Validation is also the step that determines whether delivery should be attempted at all, which makes it a decision point rather than a stage of production work.

Definition

Validation: the FDE phase in which the feasibility and potential impact of a chosen impact area are tested before the build is committed to. It ends with a decision gate that determines whether delivery is worth attempting.

Delivery Carries a Validated Project to Impact

Delivery is the final phase, where the project is carried through to impact. The source defines it as the point where the team has an impact to show for the work. Because delivery is preceded by validation, the team does not enter delivery with an open question about whether the work matters. It enters delivery already holding a decision that the impact area is worth pursuing, and its job is to carry that validated project through.

This dependency is why the order matters. If validation were skipped or moved after delivery, delivery would be carrying a project whose worth had never been tested, and the decision that validation is meant to inform would arrive too late to change anything.

Tracing a Project Through the Three Phases

Worked example

Following one engagement's state from phase to phase

A small team is starting an FDE engagement with a customer. Trace what the team holds at the end of each phase, and show what each phase hands to the next.

  1. 1
    Start of scoping

    The team knows the customer and a problem area, but no impact area has been chosen. The question is where an AI system could have the biggest impact.

  2. 2
    End of scoping

    The team, working closely with the customer, has an understanding of the problem space and a candidate impact area. There is still no finished system, and the output is understanding rather than code.

  3. 3
    End of validation

    The candidate impact area has been tested for feasibility and potential impact. The validation decision gate produces a decision about whether delivery is worth attempting.

  4. 4
    Start of delivery

    The team enters delivery already holding the decision that the impact area is worth pursuing. Its work is to carry the validated project through to impact.

  5. 5
    End of delivery

    The team has an impact to show for the work, and the working result is in the hands of the people who will use it.

Result

Each phase hands the next one a specific piece of state: scoping hands over an understood impact area, validation hands over a decision, and delivery hands over an impact.

Why Validation Gets Its Own Phase

The source makes validation a separate phase with its own deliverables and decision gate. This matters because validation has a different job from delivery and from a post-launch review. The comparison below places the three side by side. Delivery builds toward impact. A post-launch review looks back at a system that already exists. Validation sits between them and makes a decision that the other two do not make.

Decision gateDecides whether delivery isworth attemptingValidated projectCarried through to impactReviewRuns after the systemexistsTested impact areaFeasibility and potentialimpactImpact to showWorking result for usersResults filed awaySometimes without actiontested before decisioncarried throughresults may go unused
How does validation differ from delivery and from a post-launch review, and where does its decision gate sit between them?

Read the columns from left to right and the difference is clear. The validation column is the only one that ends in a decision about whether to proceed. The delivery column begins from that decision. The post-launch review column begins after the work has been finished, which is why its results can be filed away without changing what was built.

The Two Mistakes That Hide Validation

  • Common mistake

    Treating validation as a sub-phase of delivery

    What it looks like

    The team builds the system, then tests it as one step inside the build, and proceeds regardless of the result.

    Why it fails

    The source makes validation a separate phase with its own decision gate. Folding it into delivery removes the point where the team decides whether delivery should be attempted at all.

    Do this instead

    Keep validation as its own phase, completed before delivery begins, with its decision gate in place.

  • Common mistake

    Treating validation as a post-launch review

    What it looks like

    The team delivers the system and then runs a review of how it performs, with results that are sometimes filed away without action.

    Why it fails

    A review after delivery cannot inform whether delivery should happen, because the build is already complete. The source places validation before delivery for this reason.

    Do this instead

    Move the feasibility and impact test to before the build, and make its outcome a real decision about whether to pursue delivery.

  • Common mistake

    Assuming scoping is finished once a team has an idea for a system

Placing Work in the Right Phase

Applying the frame means asking which question a piece of work is answering. Work that identifies where an AI system could have the biggest impact belongs to scoping. Work that tests the feasibility and potential impact of a chosen impact area belongs to validation. Work that carries a validated project through to a working result in users' hands belongs to delivery.

In the real world

A team has an agreed impact area and is deciding whether the feasibility of that area is worth testing before any build. Writing the criteria that will judge feasibility and potential impact is work that tests the chosen focus, so it belongs to validation. Its output feeds the decision gate. Work that puts a validated result in front of the people who will use it belongs to delivery.

Best practice

When a task is unclear, ask what decision it informs. Work that informs where to focus belongs to scoping, work that informs whether delivery is worth attempting belongs to validation, and work that carries the result to users belongs to delivery.

Practice Assigning Phases

MEDIUM

A colleague says that since the team already built a prototype during scoping, the validation gate can be skipped and delivery can start immediately. Using the three-phase frame, explain what is wrong with that reasoning and name the phase where the prototype's worth should be decided.

Hints
  • Recall that scoping's output is an understanding of the problem space, not a finished system.
  • Ask what validation decides and whether delivery should be attempted at all.
  • Consider which phase tests feasibility and potential impact before delivery begins.

Key Takeaways

  • An FDE engagement moves through three phases in fixed order: early scoping, validation, and delivery.
  • Scoping asks where the work should focus, and its output is an understanding of the problem space rather than a finished system.
  • Validation tests the feasibility and potential impact of the chosen impact area, has its own deliverables and decision gate, and decides whether delivery is worth attempting.
  • Delivery begins from a validated decision and carries the project through to impact.
  • Treating validation as a sub-phase of delivery or as a post-launch review hides the decision gate and leaves the decision with nothing left to change.