Concepts / Plugin Evaluation

Plugin Evaluation

A plugin is a self-contained package of prompts, tools, and configuration for one specific task.

  • Programming

Why a Working Plugin Beats a Résumé Line

Anyone trying to show an employer that they can do forward deployed engineering work needs something concrete to point to. A résumé line saying you understand customer problems is easy to write and hard to verify. One direct way to produce concrete evidence is to build a plugin on Anthropic's platform.

The source presents this path as especially current. Anthropic's Code with Claude conference in 2026 introduced enterprise plugin templates, which cover several industries, including HR, design, engineering, investment banking, equity research, private equity, and wealth management. A plugin built in one of these verticals gives you something a reviewer can install and try, rather than a claim you have to defend from memory.

The reason plugins are attractive is that a well-built plugin is described as a microcosm of forward deployed engineering work. Even at small scale, a plugin forces you to make the same kinds of decisions a deployed engineer makes on a real customer engagement. As you read, keep one question in mind: what does a hiring reviewer need to see in order to judge the work without having to take your word for it?

Following One Plugin From Vertical to Package

The source describes making a well-built plugin as a sequence of decisions rather than a single act of writing. Follow one plugin through those decisions and notice what each decision constrains. First, you pick a vertical, meaning a specific industry or domain you understand. Second, you identify a job-to-be-done, which is one concrete task that people in that vertical need to get finished. Third, you package the prompts, tools, and configuration that let the plugin carry out that task. Fourth, you make the package self-contained enough that someone else can install it and try it in about five minutes.

What do you think happens?

Before reading on, what do you think the fourth decision in building a well-built plugin is?

Reveal answer
Answer:

Making the package self-contained enough that someone else can install it and try it in about five minutes.

The fourth step is about portability and trial. A plugin that only its author can assemble does not meet this standard, which is why the source frames self-containment as the final decision.

The order matters, and the diagram below traces it. The vertical comes first because it constrains every later choice. A job-to-be-done only makes sense inside a domain, and packaging only makes sense once you know which task it serves. Treating the plugin as a chain of decisions, rather than as a pile of prompts, is what turns a demo into something a reviewer can judge. The final stage of the diagram is the portfolio entry, which the source ties to evaluation evidence. That evidence is added to the shipped plugin, not left for a reviewer to infer.

Chosen verticalan industry you understandJob-to-be-doneone concrete taskPackaged partsprompts, tools,configurationSelf-containedpackageinstallable and tryable inabout five minutesShipped outputstrace, eval set, wrapperlogicPortfolio entrywith eval evidence areviewer can checkconstrainsdetermines what to packagemade self-containedships withevidence added for review
What are the stages a single plugin passes through, in what order, and where does evaluation evidence get added before it becomes a portfolio entry?

What Is Inside the Package

Definition

A plugin is a self-contained package of prompts, tools, and configuration for one specific task.

Each of the three parts has a distinct role. The prompts carry the instructions that tell the underlying model what to do for the task. The tools are the capabilities the plugin can call on while doing that work. The configuration holds the settings that shape how the package behaves. Because all three travel together, the plugin can be handed to someone else as one unit, rather than as a set of loose files that only you know how to assemble.

Pluginone specific taskPromptsinstructions for the modelToolscapabilities called duringthe workConfigurationsettings that shapebehaviorcontainscontainscontains
What does a plugin contain, and which parts sit inside which other parts?

The diagram shows containment only. It does not show how the parts interact at run time, because the source does not describe that interaction in detail, so this article does not claim a specific call order. What the containment view does establish is that a portfolio plugin should be readable as a package with these three named parts, so a reviewer can find each one quickly.

What a Shipped Plugin Produces

Building the package is only part of the work. The source states that a plugin produces three things: a trace, an eval set against the plugin's task, and wrapper logic that makes the plugin reliable. The source names the trace and the eval set without defining every detail of either, so treat them here as outputs you must be able to show, not as a fully specified format.

The wrapper logic is the piece the source ties to reliability. The diagram below follows the source's wording: the plugin produces the trace, the eval set, and the wrapper logic, and these outputs are what feed the evidence a reviewer can check. The word defensible in that final step reflects the source's emphasis on evidence and is not a separate technical term.

Pluginpackaged for one taskTraceoutput of a pluginEvidence entrya claim a reviewer cantrace backEval settests against the plugin'staskWrapper logicsupports reliabilityproducesproducesproducesfeedsfeedsfeeds
Which plugin outputs feed into which pieces of evaluation evidence, and how does a reviewer trace a claim back to its source?

Mistakes When Packaging a Portfolio Plugin

  • Common mistake

    Picking a vertical and task after writing the prompts

    Why it fails

    The source's order matters: the vertical constrains every later choice, and a job-to-be-done only makes sense inside a domain. Prompts written first tend to drift toward a generic demo.

    Do this instead

    Fix the vertical and the single task first, then write the prompts, tools, and configuration to serve that task.

  • Common mistake

    Shipping a plugin that only its author can assemble

    Why it fails

    The source requires that someone else can install the package and try it in about five minutes. A set of loose files does not meet that standard.

    Do this instead

    Bundle the prompts, tools, and configuration together as one unit with clear names so a reviewer can find each part quickly.

  • Common mistake

    Counting plugins instead of defending them

    Why it fails

    The source says eval evidence that lets you defend each plugin matters more than simply having many plugins in a portfolio. A plugin without evidence leaves you with little to say when a reviewer asks why it should be trusted.

    Do this instead

    Keep fewer plugins and attach the trace, eval set, and wrapper logic evidence to each one.

Planning a Portfolio Plugin

Worked example

Planning an HR plugin entry

A candidate wants a portfolio plugin that shows forward deployed engineering thinking in the HR vertical, which is one of the industries covered by the enterprise templates the source mentions. The plan must include evidence a reviewer can check.

  1. 1
    Choose the vertical

    The candidate picks HR because they understand the domain. This is the first decision, and it constrains everything after it.

  2. 2
    Name the job-to-be-done

    The candidate chooses one concrete HR task that people in that role need to finish, such as answering a single recurring onboarding question. One task keeps the plugin focused.

  3. 3
    Package the parts

    The candidate writes prompts that instruct the model for that task, lists the tools it may call, and sets configuration that shapes its behavior, then bundles all three together.

  4. 4
    Make it self-contained

    The candidate writes a short installation and trial path so that someone else can try the plugin in about five minutes.

  5. 5
    Attach evaluation evidence

    The candidate keeps the trace from runs, builds an eval set against the onboarding task, and documents the wrapper logic that supports reliability, so each claim in the portfolio entry points to a specific output.

Result

A portfolio entry with one clearly scoped HR plugin and three named evidence outputs that a reviewer can open and check. The example is generated for teaching purposes and is not drawn from the source.

Checking Your Own Portfolio Plan

MEDIUM

Choose a vertical you understand and write one job-to-be-done for it. Then list the prompts, tools, and configuration your plugin would package, and name the trace, eval set, and wrapper logic you would show a reviewer. For each output, write one sentence stating what question it lets a reviewer answer about your plugin.

Hints
  • Start with the vertical, because it constrains every later choice.
  • If a reviewer could not find one of your three parts quickly, the package is not yet readable enough.
  • If an output cannot answer a question about trust, it is not yet evaluation evidence.

Key Takeaways

  • An Anthropic plugin is a self-contained package of prompts, tools, and configuration for one specific task.
  • A well-built plugin is made by choosing a vertical, identifying a job-to-be-done, packaging the parts, and making the package self-contained enough to install and try in about five minutes.
  • A shipped plugin produces a trace, an eval set against its task, and wrapper logic that supports reliability.
  • Eval evidence that lets you defend each plugin matters more than simply having many plugins in a portfolio.