The system loop
- Improve prompts, context, retrieval and tools.
- Use memory and few-shot examples from real runs.
- Close feedback loops without changing model weights.
- This is where the source deck says most real improvement comes from.
How an agent gets better over time: from feedback loops that need no training at all, through fine-tuning, preference optimization and reinforcement learning, to systems that can improve themselves.
An agent can improve without a single gradient update. The first decision is whether the problem belongs in the fast, reversible system loop or in the slower, more expensive weights loop.
Every downstream learning method is only as good as the signal feeding it. Feedback is the raw material — and most of it is already flowing through production systems.
Thumbs, ratings, corrections and human approvals.
Edits, retries, abandonment, escalation and task completion.
Tests pass, a compiler runs, math checks out, or a transaction settles.
Supervised fine-tuning teaches format, style and domain conventions by showing the model curated examples. In the source, it is described as the cold start — not the finish line.
When there is no single correct answer, teach the model which answer is better. The source contrasts classic RLHF with direct preference optimization (DPO).
The deck presents RL as the mechanism for pushing past supervised-training plateaus, especially when the environment can verify success directly.
Skip subjective human rankings when the environment can score the answer. Run the tests, check the math, compile the code, or verify the transaction, then reward success.
Long-horizon agents may take many tool calls and thousands of tokens before anyone can score the run. The source therefore highlights frameworks that decouple rollout from gradient updates.
The three weight-loop levers are presented as a pipeline. Each stage addresses a limitation left by the one before it.
The source argues that continuously updating model weights from live production traffic is usually too risky. In practice, teams update the system around the model.
The frontier is a closed loop in which an agent identifies weaknesses, generates tasks that target them, attempts the tasks, verifies the result, and trains on what worked.
The deck closes with a practical routing guide: choose the cheapest, most reversible mechanism that actually matches the problem.