Benchmarks
Every number has a receipt.
Each result below states what was measured, the command that reproduces it in the open repo, and — because a benchmark that hides its limits is marketing — what it does not show.
The headline result
The live reuse A/B — where memory pays, and where it doesn't
A real coding agent ran through the real proxy against the same repository, with and without Kage memory. When the fact the agent needed was not in the code — tribal knowledge — memory changed the outcome. When it was in the code, memory honestly made no measurable difference.
What this does NOT show: a team-wide token saving over weeks of real traffic. That number can
only come from your own receipts — which is what kage report team accumulates.
Precision
The injection gate — measured on hostile prompts
The gate must inject nothing on content-free prompts ("fix it", "thanks"), nothing on topics the store cannot answer, and the right memory on direct questions. The acceptance bench holds all four properties at once, and is drift-checked against the shipped recall code:
npm run bench:injection --prefix mcp (use --assert-baseline in CI)
What this does NOT show: relevance on your corpus. The gate normalizes against each store's own
score distribution, and every live decision is recorded so your real rate shows up in
kage report team — not just our fixtures.
Context efficiency
History digestion — reversible, and honest about the alternative
Single-body compression on real traffic measures ~0% net — we ship that number rather than the wish. The real waste is history: old tool outputs re-sent verbatim on every turn. Opt-in history digestion reduces them to deterministic digests, with the exact original always retrievable from the content-addressed store:
npm run bench:compression --prefix mcpWhat this does NOT show: token-level savings on cached providers (a digested prefix re-keys the provider cache once, then stays stable — bytes are the measured unit here), and history digestion is off by default until you opt in.
Safety as tests
The honesty gates — enforced, not documented
npm test --prefix mcp · gate: node --test mcp/dist/vnext/phase-e-gate.test.jsWhat this does NOT show: production pilots. No design-partner pilot has run yet; the GA report says so and exits non-zero rather than pretending. Live GitHub/Stripe/identity-provider interop is fixture-proven, not exercised against production accounts.
Run them yourself
The repo is open; every bench is a committed script with its baseline recorded.