Benchmarks

Measured, not asserted — and deliberately without a headline "X% reduction" number, because reduction is a property of your data and your budget, not of the library. Two reproducible suites; full tables live in the repo (benchmarks/RESULTS.md, benchmarks/COMPARISON.md).

Suite 1 — contexel's own claims

python -m benchmarks (needs the accurate extra). Highlights:

• Determinism, 4/4 PASS — the same contract over 5,000 seeded records hashes to the same SHA-256 in three fresh interpreters with different PYTHONHASHSEED values, and shaped output is identical even when raw records carry fresh request ids and timestamps.
• Budget tightness — shaped with tiktoken, actual cost lands under budget (1,890/2,000); shaped with the default estimator it can overshoot ~0.5%. truncate_field never exceeded its cap.
• Speed — ~190k records/sec through the full contract (100k records ≈ 0.5 s); select has no measurable overhead vs a hand-written comprehension. Tracing: 49.8 ms inactive vs 173.3 ms active (3.48×) over 10k records — inactive adds single-digit milliseconds over the untraced 44.4 ms baseline; active tracing token-counts every stage boundary and is priced accordingly.
• Default tokenizer accuracy — ~4 chars/token is roughly on the mark for typical prose and code (−1.5% to +4%), but can over-estimate dense prose by ~18% and under-estimates JSON-heavy records by ~29%: install contexel[accurate] when budget precision matters.

Suite 2 — against the prior art, on real data

python -m benchmarks.competitors (needs the benchmarks extra; first run downloads the CodeSearchNet python/test split, ~30 MB). The full split — 22,176 real functions — is reconstructed into 28,047 search-tool records with genuine duplicates, plus 100 ground-truth episodes where the record the agent needed is known and guaranteed present in the raw hits, so recall isolates shaping loss, not retrieval loss. Budget: 1,500 tokens.

How to read it

• Two experiments in one table. ms @28k holds the policy constant — libraries get identical hand-written glue for whatever they lack, so timing differences are theirs. The outcome columns (Recall / Compliance / Useful) are the opposite: each library acts only through its own operations, so they show what each one actually ships.
• Recall with 0% compliance is not recall. toolz and llama-index keep the needed record by emitting 92–129× the budget — a context no window accepts. langchain-core holds the budget but, with no projection or dedupe, fills it with bloat and the needed record survives only 30% of the time.
• contexel's 15.5 ms/episode is the work, not overhead. It is the slowest budget-compliant row because it does the most: deriving relevance from ~1 KB of evidence per record on top of projecting, deduping, clipping, ranking, and budgeting. Every faster row is faster by doing less. All rows are noise next to one model call.
• The two recall columns bound the semantic concession. The hand-written row trusts the tool's score: 93% under a decent retrieval signal, 32% under a weak one. contexel's rescore derives relevance itself, so both columns read 100% — on these episodes.

Limits, stated plainly