Evaluating Customer-Facing AI Systems

Most projects do not stay exactly as they were first described. Once a team is partway through its work, a stakeholder often sees early progress and asks whether the work could also cover something adjacent. The request frequently sounds small, and the person making it may genuinely believe the team has spare capacity. What a professional does in the next few minutes matters a great deal. They decide how to think about the request, what to ask before answering, and how to talk about timing without damaging the commitment already in place.

This article follows a real negotiation between a candidate and an interviewer. A team is building an AI agent that triages internal support tickets. The customer success group now wants the same agent to handle inbound customer chat. The candidate has to decide how to respond, and the way they respond shows that scope and timeline are negotiated together rather than as two separate questions.

In the scenario, the baseline is internal ticket triage, and it is on track to be delivered in week eight. The candidate begins by stating this explicitly. Before evaluating the chat request, they restate what the team has promised and where it currently stands. This is the first step, and it comes before any judgment about the new work.

Starting from the baseline does two things. First, it protects the existing commitment from being quietly renegotiated as a side effect of a new conversation. Second, it gives both parties a shared vocabulary. When the candidate says the original delivery is on track, the interviewer knows exactly which deliverable is under discussion. The new request can then be measured against that specific deliverable rather than against a vague sense of the whole project.

Not every addition deserves the same treatment. A minor adjustment reuses existing data, existing access rules, and existing quality standards. A request that introduces new ones is a different kind of request. An addition counts as an expansion when it brings new data needs, new access or security questions, or a higher quality standard. These three checks are the core of the evaluation.

The candidate in the scenario does not simply accept or reject the chat request. Instead, they ask three diagnostic questions. How much of the chat volume would the agent need to handle on day one to be worth shipping? Is the chat data already labeled? Does the chat surface sit inside the same security perimeter as the ticket data, or would it need new identity scoping? Each question maps directly to one of the three checks in the flowchart.

Once a request is understood as an expansion, the next task is estimating its effect on the timeline. The candidate gives a specific figure: doing the chat work well adds six weeks, not three. The number matters less than the reasoning behind it. A good estimate names the dependencies that drive the duration rather than offering a single optimistic figure.

The candidate names two broad options and one warning. The warning concerns folding the chat work into the current engagement. That approach risks missing the original delivery, because the same team would now carry two sets of obligations at once. The better option is to deliver the original scope on schedule and then start a separate engagement for the chat agent. The same team and the same evaluation discipline carry over, but the new work is scoped properly on its own terms.

Phased delivery means shipping a project in stages, where each stage is built on the one before it. In the scenario, the first phase is the ticket triage agent, delivered in week eight. The second phase is the chat agent, which begins only after the first phase is secured and reaches delivery in week fourteen. The value of this arrangement is that a commitment already made is not sacrificed to a later request. The original work is complete and stable before the new work begins.

The candidate then asks which outcome the stakeholder would prefer: ticket triage in week eight with chat by week fourteen, or both together in week sixteen. The interviewer chooses the first. The candidate closes by committing to a concrete next step, drafting a follow-on engagement summary for the chat work so the stakeholder can review it. The final outcome is stated plainly: the original scope stays as it was, on its original timeline, and the expansion is accepted as separate work.

The same reasoning transfers to other projects. Consider this generated scenario. A team is partway through building a monthly sales report that is due in week six. A regional manager asks whether the report could also include live data from a new partner system.