Concepts / Defining Success Metrics for Customer Deliverables

Defining Success Metrics for Customer Deliverables

A lead's role expands gradually, supported by a senior shadow who observes and gives specific feedback.

  • Programming

Why a Prepared List Can Mislead

Imagine you are leading a customer engagement for the first time. A more experienced colleague sits beside you on the calls, and you are responsible for the questions, the schedule, and the result. Halfway through the discovery work, that colleague gives you feedback that cuts to the heart of the job: you were asking the questions you had already written down, not the questions the customer's own words were pointing toward.

This article follows a real account of a first engagement. It was a three-week effort for a mid-market insurance carrier, building a single workflow that summarizes claims for a defined class of claims. Along the way the lead learned three lessons: how leadership is handed over gradually, how to listen well enough to change course mid-interview, and how a schedule can be lost in places nobody planned to look.

What do you think happens?

Before reading on: what was wrong with the lead's questions in the second interview, even though she had prepared them carefully?

Reveal answer
Answer:

She was asking the questions she had written in advance rather than questions drawn from what the customer had just said.

The preparation itself was not the problem. A prepared list is a useful safety net, but it became a liability when it pulled the conversation away from what the customer was actually revealing.

Splitting the Work Between Lead and Shadow

A senior shadow is a more experienced person who works alongside a new lead. In this engagement the senior shadow was Theo. He attended the kickoff call and the first scoping interview. In the second interview he did not speak at all. That gap was deliberate in spirit: the lead needed room to lead, and Theo's job was to observe closely enough that his later feedback would be specific and useful.

Leadership in this setting is not a title that switches on at the start of a project. It is a responsibility that grows as the lead gains confidence, while the senior person steps back and offers corrections at moments that matter. By the third interview the lead had taken over the conversation. The table below summarizes how the split changed across the engagement.

StageLead's roleSenior shadow's role
Kickoff and first scoping interviewPresent and still finding her footingAttends the calls and observes
Second interviewRuns the questions, working from a prepared listSilent; watches closely to prepare specific feedback
Third interviewCarries the conversationSteps back and stays available
Blocked data-permissions workOwns the schedule and the escalationHelps her escalate through the sponsor with the right channel

How responsibilities shifted between the lead and the senior shadow

Mentorship is the bridge between these stages. The senior person observes, gives targeted feedback, and then helps the lead act on it. Mentorship also reaches beyond the conversation with the customer. When a data-permissions delay threatened the schedule, Theo showed the lead which channel to use: her sponsor, the internal person with authority to unblock the work.

How the Last Answer Shapes the Next Question

Adaptive questioning means changing what you ask based on what the customer has just told you. The lead's feedback from the second interview was exactly this: the questions had been prepared in advance, when they should have been drawn from the customer's last answer.

Adaptive questioning is a skill that has to be practiced rather than simply understood. The lead did not stop preparing. She still walked into each conversation with a plan. What changed was her willingness to set the plan aside when a customer's answer opened a more useful thread. The account describes her working on this one habit through the rest of the engagement, which is how a single piece of feedback became a durable change.

The flow below is an illustrative model of this decision, not a procedure taken from the source. At each answer, the interviewer asks whether the answer points toward a useful follow-up. If it does, the next question is built from that answer. If it does not, the interviewer returns to the next topic on the prepared list. In both branches, the interviewer then listens to the next answer and repeats the check.

Customer answersUseful follow-up?Does the answer open athread?Build next questionDrawn from the answerListen to next answerNext prepared topicReturn to the listcheckyesnorepeat
How does one customer answer branch into different follow-up questions, and which path do I take next? This is an illustrative model of the decision, not a source procedure.

Building a Follow-Up From a Real Answer

Worked example

Choosing the next question from a customer's words

A customer says that adjusters often re-key claim details by hand because the intake form and the claim system do not match. The prepared list's next topic is 'preferred output format.' Should the interviewer ask that question next?

  1. 1
    Identify what the answer reveals

    The customer has described a real pain point: manual re-keying caused by a mismatch between two systems.

  2. 2
    Check whether it opens a useful thread

    The answer points toward how claims move through the workflow, which bears directly on what the summary must contain and what success looks like.

  3. 3
    Build the follow-up from the answer

    Ask which fields are re-keyed most often and how long that step takes per claim. This question comes from the customer's words, not from the list.

  4. 4
    Return to the list only when the thread is exhausted

    Once the re-keying thread has been explored, the interviewer can go back to the output-format topic and ask it with fuller context.

Result

The next question is built from the re-keying answer. The prepared topic is deferred, not discarded.

Defining the Metric and Its Threshold

A success metric is the measurable quantity that tells you whether the engagement is delivering what the customer needs. In this engagement the metric was minutes saved per processed claim. It asks how much faster a claim is handled once the workflow is in place, and it gives the customer and the team a shared number to discuss.

Defining the metric is only the first step. The team also needs a threshold: the level the metric must reach before the customer will accept the result. Here the threshold was thirty percent. A metric must be defined in terms the customer recognizes, because a percentage gap is easy to state and easy to check. That lets everyone agree on how far the work still has to go before any new request is discussed.

Worked example

How far the work still has to go

During the first week of validation the metric reached twenty-three percent against a thirty percent threshold. How much is still missing?

  1. 1
    Write down the threshold

    The customer will accept the result only at thirty percent, so the threshold is the target.

  2. 2
    Write down the measured value

    The metric reached twenty-three percent during the first validation week.

  3. 3
    Subtract the measured value from the threshold

    $30\% - 23\% = 7\%$. The remaining gap is seven percentage points.

Result

The team still needs seven more percent. The customer later asked for exactly this remaining amount, which is a useful check on the arithmetic of the gap.

Following the Schedule Through the Engagement

The engagement stretched across two calendar months, February and March. Laying the key events out in order shows how leadership and schedule interacted. The timeline below follows the sequence the account describes, from the kickoff through the validation extension that followed the first delivery.

Kickoff and scopingSenior shadow attendsSecond interviewPrepared questions usedThird interviewLead carries theconversationFirst slice reachesstagingTwo days lateMetric reaches 23%Threshold is 30%RevisionsAbout three days ofengineeringValidation extensionLasts two weeks
How does a small gap found late cost more total time than fixing it early, and how does the delay build up across later tasks? Read left to right and notice where the lead's role changes.

Read the timeline with two questions in mind. First, where does the lead's role change? In the first two interviews the senior shadow is present and the lead is still finding her footing. By the third interview the lead is carrying the conversation. Second, where does the schedule slip? The later events in March test a different skill: keeping a schedule and a success target in view when the work hits obstacles outside the conversation room.

Why the Extension Outlasted the Fix

Schedules in customer engagements rarely fail where the work is hardest. The first slice reached the customer's staging account two days after the planned date because of a data-permissions delay. That delay was real but solvable, and the lead resolved it by escalating through her sponsor with Theo's help.

The larger delay came from the revision cycle. The model and prompt changes needed to close the gap to the threshold took about three days of actual engineering work. The validation extension, however, lasted two weeks. Most of that time went to getting revised work onto the customer's staging environment, not to the revisions themselves. The lead named this as the single biggest lesson of the engagement.

The same logic explains why a gap found late is costly. A metric gap that is discovered during validation sends the work back through revision and back through delivery. Each pass adds time on top of the fix itself, so the remaining work can take longer than the fix.

Mistakes New Leads Make

  • Common mistake

    Treating the prepared question list as the conversation

    What it looks like

    Asking the next scripted question even after the customer has just described a problem that deserves follow-up.

    Why it fails

    The list is a safety net. When it pulls the conversation away from what the customer is revealing, the interview stops learning anything new.

    Do this instead

    Check each answer for a useful follow-up. Build the next question from that answer, and return to the list only when the thread is exhausted.

  • Common mistake

    Defining a metric the customer does not recognize

    Why it fails

    Without a shared number, the team and the customer cannot agree on how far the work still has to go.

    Do this instead

    Express the metric in terms the customer uses, and set a threshold that both sides can check.

  • Common mistake

    Estimating the schedule only from the technical work

    What it looks like

    Planning a validation extension based on how long the revisions take, while ignoring how long delivery to the customer's staging environment takes.

    Why it fails

    In the account, the revisions took about three days but the extension lasted two weeks, mostly spent on getting work onto staging.

    Do this instead

    Include the delivery path, including access and environment steps, in the schedule.

  • Common mistake

    Trying to lead alone without using the senior shadow's observations

    Why it fails

    The senior shadow's job is to observe closely and give specific feedback. Ignoring that feedback means the same habit repeats.

Practice Planning a Follow-Up

MEDIUM

A customer tells you that their claims team has to check each summary against the original claim notes before anyone trusts it. The prepared list's next topic is 'data retention rules.' Describe the follow-up question you would ask next, and explain how it is drawn from the customer's last answer. Then state how you would measure the remaining gap if the metric reached 21 percent against a 30 percent threshold.

Hints
  • Start from the phrase 'check each summary against the original claim notes.' What does that reveal about trust and accuracy?
  • A follow-up question can ask how often the checking happens, or what the team looks for when it checks.
  • Use the same subtraction as the worked example: threshold minus measured value.

Key Takeaways

  • A lead's role grows gradually. A senior shadow observes, gives specific feedback, and helps the lead act on it, including outside the customer conversation.
  • Adaptive questioning means building the next question from the customer's last answer, and returning to the prepared list only when that answer does not open a useful thread.
  • A success metric needs a threshold the customer recognizes. The remaining gap is the threshold minus the measured value, such as 30 percent minus 23 percent, which leaves 7 percent.
  • Schedule delay often sits in the delivery path. Revisions that took about three days were followed by a two-week extension, mostly spent getting work onto the customer's staging environment.