An agent can call every tool correctly, land on the right final answer, and still have made fifteen LLM calls when three would do, or hallucinated a tool result it got lucky with. Scoring only the output hides exactly the problems that will surface as cost and reliability issues at scale.
Agent testing and evaluation scores not just whether an agent's final answer was correct, but the entire trajectory it took to get there — the chain of reasoning, tool calls, and intermediate steps. This matters because two runs of the same task can take completely different paths and both be correct, or both be wrong for entirely different reasons; a single output-only score can't tell those cases apart. Production-grade evaluation works at three levels — end-to-end (did the task succeed), trajectory-level (was the path efficient and sound), and component-level (which specific tool, retriever, or sub-agent broke) — scored across five core dimensions: task completion, accuracy, hallucination rate, cost, and latency/reliability.
Evaluating a plain LLM call is simple: one input, one output, score the answer and move on. An agent produces a trajectory instead — a chain of reasoning, tool calls, and intermediate steps unfolding over many turns before it ever reaches a final answer.
That changes what "good" means. Two runs of the same task can take completely different paths and both be correct, or both be wrong for different reasons entirely. An agent can pass on the final answer while making three times the necessary tool calls, or fail while having reasoned about the problem correctly. Output-only scoring sees neither.
End-to-end asks the simplest question: did the task succeed? This is task completion — the clearest signal of whether the agent actually worked, though it depends entirely on how "done" is defined for the task at hand.
Trajectory-level asks whether the path was efficient and sound — not just whether the destination was right. Component-level goes further still, isolating which specific retriever, tool, or sub-agent in a longer chain actually broke. Trace-based evaluation connects a failing score to the exact span that caused it, rather than leaving a team to guess.
Write down a specific metric and threshold for each of these before building automated scoring — they anchor every evaluation decision that follows.
What task completion rate counts as acceptable? What cost per task makes the agent economically viable? What latency will your users tolerate? Answering these in writing, before building automated scoring, is what keeps an evaluation program anchored to something real rather than a vague sense of "seems fine."
Did the agent achieve the goal the user actually asked for?
Are the facts, values, and outputs the agent produced actually correct?
How often does the agent state something ungrounded as fact?
What does one successful task actually cost in tokens and tool calls?
How long does a task take, and how consistently does it succeed?
Trajectory evaluation captures the full nested trace of model calls and tool calls, then scores it against a reference path — tool-call accuracy, step count against the minimum needed, whether the agent looped or backtracked, and whether it recovered gracefully from a wrong turn rather than compounding it.
This is where cost and latency problems actually originate. An agent that passes on the final answer while taking five times the necessary steps looks fine in a pass/fail dashboard right up until the monthly bill or the p95 latency chart tells a different story.
LLM-as-judge is the default scorer across most evaluation platforms: give a model the agent's output and a rubric, and let it grade dimensions that resist purely programmatic scoring — helpfulness, tone, whether an explanation actually makes sense.
It comes with real, well-documented biases: a preference for longer answers, a sensitivity to answer position, and self-preference toward outputs that resemble its own style. It's also non-deterministic, and a full model call per judgment is too expensive to run on every production turn. Research testing grading scales found a 0–5 scale with explicit criteria at each level aligns best with human judgment — better than binary pass/fail or a noisy 10-point scale. Use it calibrated, offline, on a sample — not as an unquestioned oracle on every turn.
Begin with 50 to 100 test cases covering happy paths, known edge cases, and any failure mode already seen in the wild. Grow the set continuously by folding in production regressions as they surface — aim for 500 or more within the first quarter.
A hundred carefully curated cases with precise expected trajectories beat a thousand auto-generated ones with vague expected outputs. Run the golden dataset on every pull request, scoring task completion and accuracy programmatically wherever possible, and reaching for a calibrated LLM-as-judge only for the dimensions that genuinely resist a deterministic check.
No single tool covers everything — most teams end up combining a general-purpose eval harness with something purpose-built for trajectory scoring.
General-purpose harnesses — LangSmith, Braintrust, Phoenix, DeepEval — cover LLM-as-judge, structured-output checks, and everyday app evals with sensible defaults. Purpose-built trajectory tools layer on top for the agent-specific case. MLflow's tracing captures the full execution trace and scores tool selection, plan quality, and execution efficiency across it. Standard benchmarks — AgentBench, tau-bench, SWE-Bench — offer external reference points, though they measure general capability more than your specific production workload.
| Tool / benchmark | Best for | Coverage |
|---|---|---|
| LangSmith / Braintrust / Phoenix | Everyday LLM-app evals, CI integration | General harness |
| DeepEval | Prebuilt evaluators, consistent scoring harness | General harness |
| agentevals | Trajectory-specific scoring against a reference path | Trajectory specialist |
| MLflow (Agent GPA) | Full trace capture, tool selection, plan quality | Trajectory specialist |
| promptfoo | CI-gated evals plus red-teaming for 50+ vulnerability types | Red-teaming / CI |
| AgentBench / tau-bench / SWE-Bench | External capability benchmarks, not production-specific | Reference benchmark |
A prompt change that improves task A can quietly break task B. Without running the full test suite on every release, these regressions accumulate silently, and the first sign is a spike in user complaints rather than a failing test.
Cheap per-turn classifiers can label production traffic inline in well under a hundred milliseconds — far cheaper than a full LLM-as-judge call on every turn — feeding labels back into the eval set, fine-tuning data, and reward signals. Tracking cost-per-task as a first-class metric, and plotting accuracy against cost for every agent version, turns "the bill was higher this month" into a chart you can actually act on before it happens again.
A held-out benchmark run once before launch tells you how the agent did on tasks you already knew about. It says almost nothing about the traffic you haven't seen yet — whether it looped before answering, called the wrong tool and quietly recovered, or left a real user frustrated three turns in.
The teams that keep agents reliable treat evaluation the way mature engineering teams treat tests: a golden dataset that runs on every pull request, trajectory and component-level scoring alongside the final answer, and production monitoring that catches drift before it becomes a support queue.
Because an agent's final answer can be correct while the path to it was wasteful, risky, or lucky — extra unnecessary tool calls, a hallucinated tool result that happened to return something useful, or a loop that eventually broke out. Output-only scoring hides all of these until they surface later as cost, latency, or reliability problems at scale.
Trajectory evaluation scores the step-by-step path an agent takes to reach a solution rather than just the final answer — checking tool-call accuracy, step count against the minimum required, whether the agent looped or backtracked, and whether it recovered cleanly from an error.
With calibration, yes, for the dimensions that resist deterministic scoring. But it carries known biases — favoring longer answers, sensitive to answer position, and prone to self-preference — plus non-determinism and real per-call cost. Use it offline on a calibrated sample with a well-defined 0–5 rubric rather than as an unquestioned oracle on every production turn.
Start with 50 to 100 carefully curated cases covering happy paths, edge cases, and known failure modes, then grow continuously by adding real production regressions — aiming for 500 or more within the first quarter. A hundred precise cases with exact expected trajectories beat a thousand vague, auto-generated ones.
By running the full golden test suite on every pull request rather than only before major releases, tracking cost-per-task and accuracy-vs-cost curves per agent version, and using cheap per-turn classifiers to label live production traffic — feeding those labels back into the eval set instead of waiting for a support-ticket spike to reveal the problem.
Standard benchmarks like AgentBench, tau-bench, or SWE-Bench measure general agent capability against a fixed, public task set — useful as an external reference point, but they rarely reflect your actual production workload. A golden dataset built from your own real tasks and observed failure modes is what production evaluation should be anchored to.
Define your five core metrics and thresholds first, build a 50-case golden dataset this week, and wire trajectory scoring into every pull request before your next release.