Agent observability is the discipline of tracing every model call, tool execution, and reasoning step as a structured, replayable trace — because agents fail in ways that look like success: well-formed but wrong outputs, redundant tool calls, a plan that quietly wandered off course.
Agent observability is complete visibility into what an AI agent actually did in production — every model call, tool selection, decision chain, memory retrieval, and agent-to-agent handoff, captured as structured, hierarchical spans you can replay. It goes beyond traditional monitoring, which can confirm a service is up but can't confirm the agent picked the right tool, passed the right arguments, or stayed on its original plan. The emerging standard unifying it is OpenTelemetry's GenAI semantic conventions — a vendor-neutral vocabulary of gen_ai.* span and metric attributes that any instrumentation library can emit and any backend can ingest, so a team instruments once and can swap observability platforms without re-instrumenting every trace point.
Traditional APM tracks a request hitting middleware, querying a database, and returning a response — deterministic, and measured in milliseconds. An agent receives a prompt, reasons in loops, calls tools, evaluates results, and may call more tools before producing an output. The same input can trigger a completely different tool sequence on the next run, and duration can range from 500 milliseconds to several minutes for a multi-step task.
Worse, agents fail in ways that look like success: a well-formed, confidently-written answer that's simply wrong, a redundant string of tool calls that still lands on the right result, an action that's syntactically valid but semantically off. A dashboard confirming "the agent service is up" tells you nothing about any of that.
A single conversation turn can generate seven or more nested spans; multiply that across five to ten turns and it's clear why production agents generate ten to a hundred times more telemetry than a traditional API.
A minimum viable agent trace schema captures a distinct span for each kind of thing that can go wrong — so the surface where a failure shows up is also the surface where it gets fixed. Skip tool-execution spans, for instance, and a tool silently returning stale cached data becomes invisible: the LLM wasn't wrong, the tool wasn't throwing errors, but the agent still acted on data that was quietly out of date.
Model name, token counts, latency, and finish reason for a single model invocation.
Tool name, arguments, raw output, duration, retry count, and error state.
What was retrieved, from where, and whether it was actually relevant to the step.
The plan, the decision made, and how control passed between agents or steps.
Before a shared standard, switching observability backends meant re-instrumenting every trace point by hand. OpenTelemetry's GenAI semantic conventions changed that: a standard set of gen_ai.* span and metric attributes that any instrumentation library can emit and any OTLP-compatible backend can ingest.
Adoption moved fast once the spec landed — major agent frameworks now emit OTel-compliant spans natively or via a thin instrumentation package, and major backends read the convention directly. The practical result: instrument once, and swap exporters instead of rewriting instrumentation every time the observability stack changes.
By default, most instrumentation captures only metadata — model name, token counts, latency, finish reason — deliberately leaving out prompt content and tool arguments, since those can carry sensitive data. Enabling content capture populates full prompt messages, system instructions, tool schemas, and tool results, which is far more useful for debugging but needs to be handled deliberately, not by default.
Most production setups are multi-agent by design: a planner delegates to specialist agents, which call tools and other services. Observability has to show how a single user request moves through that entire chain, not just what one agent did in isolation.
The technique is context propagation — trace context passed along whenever an agent calls another agent or a tool, so opening a trace shows the root agent span at the top, with child spans for routing, planning, specialist agents, and tool usage nested underneath. That view shows exactly which agent took responsibility at each step, even across queues and async workers.
Agent cost often hides in sub-agents, retries, and external API calls rather than the top-level request — which is why cost attribution needs to reach per-run, per-prompt, per-sub-agent, and even per-customer, not just per top-level agent. One documented production case burned over thirteen thousand dollars in a single month before prompt caching, model routing, and a plan-and-execute redesign cut it by more than ninety percent.
Pricing models add their own subtlety: some platforms only bill LLM spans, leaving tool, retrieval, and embedding spans free — which means a highly agentic system with many tool calls but relatively few model calls can be dramatically cheaper to observe than a flat per-span pricing model would suggest.
Tracing itself has largely commoditized around the OTel standard — what differentiates platforms now is how well they surface failing runs and turn production traces into the next test cycle.
Grouping the options by how they run makes the choice clearer: self-hosted stacks like Grafana Tempo or open-source Phoenix suit teams that want full data control; managed SDKs like Datadog or Honeycomb trade some control for less operational overhead; purpose-built agent platforms like Braintrust or LangSmith fold tracing directly into an evaluation and release workflow.
| Platform | Best for | Model |
|---|---|---|
| Datadog / Honeycomb | Managed, AI-aware APM with native GenAI span support | Managed SDK |
| Arize Phoenix | Open-source, OTel-first tracing plus custom evaluators | Self-hosted |
| Langfuse | Open-source trace store with prompt management built in | Self-hosted |
| LangSmith | Native fit for LangChain / LangGraph agent stacks | Managed SDK |
| Braintrust | Trace-to-eval workflow, production failures become test cases | Trace-to-eval platform |
| Sentry | Agent traces unified with errors, replays, and infra traces | Managed SDK |
The recurring story across teams that get this right is the same: the first time a production failure gets diagnosed in minutes from a trace instead of hours spent grep-ing through unstructured logs, the value of investing in tracing early stops being an abstract argument.
Treat every user request as a single trace with spans for planning, model calls, tool calls, and downstream services from the very first deployment — not as an instrumentation project that gets added only after the first serious outage forces the question.
LLM monitoring measures individual model calls — latency, tokens, errors. Agent observability tracks the complete agent cycle: multi-step reasoning, tool execution, agent-to-agent handoffs, and how individual calls combine into a full workflow. LLM monitoring can tell you a call took 2 seconds; agent observability tells you why the agent made eight model calls when three should have sufficed.
OpenTelemetry's GenAI semantic conventions define a standard vocabulary of gen_ai.* span and metric attributes for LLM calls, tool invocations, and agent operations. Instrumenting against this standard means an agent's telemetry can be exported to any OTLP-compatible backend, so switching observability platforms means swapping an exporter rather than re-instrumenting the entire codebase.
LLM call spans (model, tokens, latency, finish reason), tool execution spans (tool name, arguments, output, duration, retries), memory operation spans (what was retrieved and from where), and agent orchestration spans (the plan, the decision, and how control passed between steps or agents). Missing any one creates a blind spot that only surfaces as a customer complaint.
Through context propagation: trace context is passed along whenever one agent calls another agent or a tool, so a single trace shows the root agent span at the top with nested child spans for routing, planning, specialist agents, and tool usage — even across queues and async worker processes.
By capping retries at the instrumentation level so a single failure cascade doesn't generate hundreds of spans, using tail sampling that keeps all error traces but only a fraction of successful ones, and understanding their platform's pricing model — some bill only LLM spans, making tool-heavy agents cheaper to observe than a flat per-span price would suggest.
Not by default. Standard OTel exporters will happily ship full prompt and tool-argument content to an observability backend if content capture is enabled. Most instrumentation captures only metadata — model, tokens, latency — by default for this reason; enabling full content capture should come with explicit masking or redaction for sensitive fields.
Instrument the four core span types against the OpenTelemetry GenAI conventions from day one, and pick an observability platform by deployment model — not just by feature checklist.