Collecting User Feedback

Imagine you are responsible for an AI application that answers customer questions. You want to know three things about each reply. Is it safe? Does it address what the customer actually asked? Are the facts it states correct? A single score is unlikely to capture all three. Each question pulls toward a different way of checking the output, and choosing that way is the task this article addresses.

A criterion is the quality you care about. A method is the mechanism that produces a judgment about that quality. The mapping between them is not one to one by default, so the useful question is which mechanism suits which quality. The source names three criteria that show how different they can be.

Toxicity detection can be handled by a small, specialized toxicity classifier, which is a model built for that single narrow job. Relevance, meaning how closely the response matches the user's original question, can be measured with semantic similarity. Factual consistency, meaning whether the response agrees with the whole context it was given, can be judged by an AI judge, which is a model asked to assess the output.

Choosing a method is not always a single either-or decision. The source notes that you can mix and match methods for the same criterion. One pattern it describes pairs a cheap classifier, which produces a low-quality signal on every item in your data, with an expensive AI judge, which produces a high-quality signal on only a small fraction, such as 1 percent.

The reason this works is cost. A cheap automatic scorer can cover all outputs, while the costly AI judge checks a small sample, which keeps the total cost manageable. Follow a single output through the process in the diagram below. Every output passes through the cheap scorer. Only the outputs selected for the sample reach the AI judge.

When a model is asked to choose among several options, it does not only pick one. Internally it assigns a likelihood to each candidate. Logprobs are the logarithms of those probabilities, and they let you inspect those likelihoods. The source's guidance is direct: when logprobs are available, use them. They can measure how confident a model is about a generated token, which is especially useful for classification tasks.

The contrast between two situations is the core of the idea. A flat spread of probability across the candidates signals hesitation, while one dominant probability signals conviction. Because that confidence is visible in the numbers, you can use it as an evaluation signal without asking anyone to read every output.

Automatic metrics should carry most of the load, since they are cheap and can run continuously. The source is equally clear, however, that you should not be afraid to fall back on human evaluation, even after an application is in production. Having human experts manually judge a model's quality is a long-standing practice in AI, and it remains valuable because open-ended responses are hard to score automatically.

Many teams treat human evaluation as a North Star metric that steers development. A practical routine is to have experts review a subset of each day's outputs. This daily sample can reveal changes in performance or unusual patterns in how people use the application. The source gives one concrete case: LinkedIn built a process to manually evaluate up to 500 daily conversations with its AI systems.

The right setup also depends on when you are evaluating. During experimentation you often possess reference data, meaning known good answers you can compare outputs against. In production that reference data may not be immediately available. Production has a different advantage: you have real users, whose behavior and feedback are evidence that a lab setting cannot supply. Feedback that agrees with your automatic scores builds confidence in them, while feedback that disagrees points to a criterion your metrics may be missing. The source states that the collection techniques themselves are covered in Chapter 10, so this article only explains why feedback matters when choosing evaluations.