Evaluating Agent Performance with Benchmarks

Suppose an agent is asked to plan a task, carry it out using a set of tools, and report back. The answer it gives looks fluent and confident. How would you know whether the agent actually did its job well? A confident final answer hides the steps that produced it, and a polished report can sit on top of a plan that was wrong from the start.

The starting point for answering that question is not a score on a leaderboard. It is a list of the specific ways the agent can go wrong. In this sense, evaluation is the work of detecting failures. This article explains the main categories of agent failure, shows how a tool use failure differs from a badly formed or badly valued tool call, and walks through a plan call by call to find where it breaks.

The more complex the task an agent performs, the more places there are where something can break. A single model answering one question has relatively few points of failure. An agent that decides which steps to take, chooses tools, passes arguments to them, and works through many actions has many more. This is why agents need their own treatment in evaluation rather than being judged only by the standards that apply to any AI application.

Beyond the failure modes that every AI application shares, agents have failures that come from three sources: planning, tool execution, and efficiency. The diagram below arranges these as a family tree under the heading of agent failure modes. The three branches are siblings, and each one can be studied and measured on its own.

Planning is the stage where an agent decides which steps to take and which tools to use. The source is direct about how this goes: planning is hard and can fail in many ways. Of those ways, the most common is tool use failure.

The planning category breaks down further. Tool use failure sits under planning, and three specific ways of misusing tools sit under tool use failure. Reading the hierarchy from the top down gives you a checklist to run when you review a plan. Each of the three leaves is a distinct mistake that can appear in a plan. They are worth separating because they are caught in different ways, and because the fix for one does not automatically fix the others.

The first leaf is a plan that names a tool not in the agent's inventory. The second and third leaves both involve a valid tool that is called with bad arguments, and these two are easy to confuse. The next section sets them side by side.

To make the three planning errors concrete, consider a small inventory with two tools. The first tool is get_weather, which takes a city and a date. The second is book_hotel, which takes a city, a check-in date, and a number of nights. Every call the agent plans can be compared against this inventory.

The three columns in the comparison show the distinction the source draws. In the first column, the plan reaches for a tool that does not exist in the inventory. In the second, the tool exists, but the call is structurally wrong because a required parameter is absent. In the third, the call has the right structure, and only the content is wrong. A valid tool called with invalid parameters is a structure problem. A valid tool called with incorrect parameter values is a content problem.

The distinction between the two parameter-related errors affects how each one is detected. The table below summarizes the difference in terms of what is wrong, how it is found, and what it takes to confirm it.

Now apply the distinctions to a whole plan. The task is to check the weather in Lisbon on June 10, 2025 and then book a hotel there for a two-night stay starting that day. The inventory contains get_weather and book_hotel, as described above. The agent produces four planned calls, in order. The goal is to trace the plan call by call and find the first place where it would fail.

Identifying failure modes is only half of evaluation. The source's instruction is to identify an agent's failure modes and then measure how often each one happens. The claim that tool use is the most common planning failure is a claim about frequency. It can only be checked by counting occurrences across many runs. A single failed run tells you that a failure is possible; a count across runs tells you how much it matters.

Measurement is also what makes the three categories useful. Planning, tool execution, and efficiency are separate branches, so each one can be counted separately. An evaluation that reports only a single overall score would hide which branch is responsible for the failures. Counting per category tells you where the agent needs work.

The source points to external resources for this kind of measurement, including the Berkeley Function Calling Leaderboard, the AgentOps evaluation harness, and the TravelPlanner benchmark. It also mentions a simple benchmark the author built to illustrate these failure modes. The source does not describe how these resources work in detail, so treat them as places to look rather than as descriptions of their methods.