Get access

L7 Performance and trust

Prompt caching

Keeps the start of every request identical so the provider can reuse it at a lower price.

Settled practice

Why it matters

Every turn resends the instructions, the tools and the whole conversation so far. When that repeated start stays exactly the same the provider reuses it far more cheaply and answers sooner, and one changing detail near the top throws the saving away on every turn.

What to build

Freeze the prefix, push volatile data below

The default the research recommends. The reasoning and the code behind it are in the app.

What it takes, by kind of agent

  • Conversational assistantMust-have
  • CodingMust-have
  • Deep researchMust-have
  • Browser/GUIMust-have

What these levels mean

How well it is studied

Harnesses checked
30
Checked facts behind it
327

See the 327 checked facts and how each harness does it

The facts, the code lines they come from and the rules for this topic are in the app. Sign in with Google; new accounts are approved by hand.

See the evidence in the app

Also in L7 Performance and trust

  • ObservabilitySettled practice

    Records what the agent is doing and what it has spent, in one count that both the display and the spending limits read.

    • Assistant Must-have
    • Coding Must-have
    • Research Must-have
    • Browser Must-have
  • StreamingOpen problem

    Delivers the agent's output live, safely and without loss, even when the connection drops.

    • Assistant Must-have
    • Coding Must-have
    • Research Must-have
    • Browser Must-have
  • ConcurrencyOpen problem

    Runs work in parallel where it is safe, one at a time where it must be, and always within limits.

    • Assistant Must-have
    • Coding Must-have
    • Research Must-have
    • Browser Must-have
  • System promptSettled practice

    Builds the agent's standing instructions from fixed sections, keeps everything that changes at the end, and fences off outside text.

    • Assistant Must-have
    • Coding Must-have
    • Research Must-have
    • Browser Must-have
  • SandboxingSettled practice

    Keeps the agent's commands, network traffic, files and secrets inside walls, and treats outside text as data, not orders.

    • Assistant Must-have
    • Coding Must-have
    • Research Must-have
    • Browser Must-have
  • Eval harnessSettled practice

    Tests the real agent loop with scripted and recorded model replies, scores open-ended work, and blocks changes that make results worse.

    • Assistant Must-have
    • Coding Must-have
    • Research Must-have
    • Browser Must-have