How the research works

Where the facts come from

We read the source code of 32 agent harnesses, such as the tools that run coding agents, research agents and assistants. Each open-source harness is read at one pinned version: a single commit, noted by its short code. A fact stays true to that version, and anyone can open the same lines. A harness whose code is closed is studied from its official documentation instead, and it is marked that way.

Every fact is tied to code

A fact is a short statement, such as how a harness decides when to shrink its context. We count it only when it points at a file and a line, or at the official docs for closed products. Facts that cannot be tied to a source are kept as leads and are never counted. Today that is 11,556 facts across 33 topics. The app shows each fact beside the line it came from.

Review rounds

The research moves in rounds (23 so far, last updated 2026-10-28). In a round we read new code, re-open earlier facts and correct the ones that were wrong, out of date or too strong. We study each harness from outside, independently: no harness author writes or signs off a verdict, and we say where a harness falls short as well as where it does well.

What the verdicts mean

Each topic gets a plain verdict for your kind of agent. There are no scores, rankings or meters.

Must-have

An agent of this kind needs it. Without it the agent is likely to fail in ways the research has seen.

Recommended

Most agents of this kind are better with it, and few good reasons exist to skip it.

Optional

Useful in some designs. Choose it when your agent has the problem it solves.

Doesn’t apply

This kind of agent does not need it, for example because it has no browser or no long sessions.

How settled a topic is

Settled practice

Harnesses that handle it well do it in much the same way.

Emerging

A few harnesses do it and the approach is still changing.

Open problem

Nobody has a convincing answer yet. We say so rather than guess.

Topics in layers

The 33 topics are grouped into 8 layers, from L0 (foundations such as errors and failover) up to L7. See the whole map.

What is on this website, and what is in the app

This website shows the map, the verdicts and the counts. The checked facts, the code lines, the rules for each topic, comparisons between harnesses and the assistant that answers your questions are in the app. You sign in with Google, and new accounts are approved by hand.