Hard limits
Maximum tokens, model calls, tool calls, wall-clock time, and external service spend.
Agent Economics
Make agent plans cost-aware
An agent plan is also a resource allocation decision. Every branch, model call, retrieval, and tool invocation consumes money and time, so the planner needs explicit budgets and trade-offs.
Answer in brief
Give the planner a hard budget, estimate the cost of each action, route simple steps to cheaper deterministic or smaller-model paths, and stop when the expected value of another step is lower than its cost and risk.
Budget model
Use these dimensions to make the architecture decision explicit and testable.
Quality target
Minimum acceptable confidence or task score.
Risk premium
Higher-risk actions require stronger evidence and may justify more expensive verification.
Escalation
When the budget cannot achieve the target, hand off rather than silently lowering quality.
Optimization levers
Use these dimensions to make the architecture decision explicit and testable.
Routing
Use rules or a small model for classification and reserve larger models for ambiguous reasoning.
Caching
Cache deterministic retrieval, embeddings, and stable tool responses with clear invalidation.
Parallelism
Run independent checks concurrently when it reduces latency without duplicating expensive work.
Early stopping
Stop after sufficient evidence, not after a fixed number of reasoning steps.
Measurement
Use these dimensions to make the architecture decision explicit and testable.
Cost per completed task
Include model, retrieval, tool, infrastructure, and human-review cost.
Useful latency
Measure time to an accepted result, including retries and approvals.
Quality-cost frontier
Compare configurations at matched quality rather than comparing cost alone.
Waste rate
Track abandoned branches, duplicate calls, unused retrieval, and repeated failures.
Production checklist
Use this as a design-review gate before release.
- Set budgets before execution.
- Route deterministic work away from large models.
- Price every tool call.
- Use confidence-calibrated stopping.
- Escalate when budget and quality conflict.
- Review cost by successful outcome, not request.
Continue learning
Place this pattern inside a broader agent architecture and operating model.
AI agent tutorial library
Browse memory, planning, frameworks, reliability, safety, and observability.
Open the tutorial library →AI agent guide
Start with agent features, design patterns, benefits, challenges, and industry applications.
Open the guide →