L7 Performance and trust
Observability
Records what the agent is doing and what it has spent, in one count that both the display and the spending limits read.
Why it matters
Operators need to see a run's steps, its cost and why it stopped. If the cost shown and the cost enforced come from different counts, a runaway run can overspend while the screen looks fine.
What to build
One priced run ledger that the cap reads
The default the research recommends. The reasoning and the code behind it are in the app.
What it takes, by kind of agent
- Conversational assistantMust-have
- CodingMust-have
- Deep researchMust-have
- Browser/GUIMust-have
How well it is studied
- Harnesses checked
- 30
- Checked facts behind it
- 422
See the 422 checked facts and how each harness does it
The facts, the code lines they come from and the rules for this topic are in the app. Sign in with Google; new accounts are approved by hand.
Also in L7 Performance and trust
- Prompt cachingSettled practice
Keeps the start of every request identical so the provider can reuse it at a lower price.
- Assistant Must-have
- Coding Must-have
- Research Must-have
- Browser Must-have
- StreamingOpen problem
Delivers the agent's output live, safely and without loss, even when the connection drops.
- Assistant Must-have
- Coding Must-have
- Research Must-have
- Browser Must-have
- ConcurrencyOpen problem
Runs work in parallel where it is safe, one at a time where it must be, and always within limits.
- Assistant Must-have
- Coding Must-have
- Research Must-have
- Browser Must-have
- System promptSettled practice
Builds the agent's standing instructions from fixed sections, keeps everything that changes at the end, and fences off outside text.
- Assistant Must-have
- Coding Must-have
- Research Must-have
- Browser Must-have
- SandboxingSettled practice
Keeps the agent's commands, network traffic, files and secrets inside walls, and treats outside text as data, not orders.
- Assistant Must-have
- Coding Must-have
- Research Must-have
- Browser Must-have
- Eval harnessSettled practice
Tests the real agent loop with scripted and recorded model replies, scores open-ended work, and blocks changes that make results worse.
- Assistant Must-have
- Coding Must-have
- Research Must-have
- Browser Must-have