Everything between query and answer.

Every backend ran inside one shared harness on a random SimpleQA sample (seed 42). Each got the same questions and its default settings. Answers were drawn only from content the tool actually retrieved, then graded against the gold answer by Claude Sonnet 4.6. Audit logs record every question, every tool's answer, and the verdict - all runs passed a leak check (0 cases of a tool credited for an answer it did not retrieve).

Two ways to read it. Single-shot (one query, no reformulation) measures raw retrieval - Sofya is #4. Agentic (the loop, matching the methodology vendors publish) measures real-world use - Sofya is #2 and tied for the lead. Both are shown above.

About the WebSearch baseline. * WebSearch is the web search built into Claude itself, not a search API you can buy. We include it as a reference point, and it wins single-shot (90% vs our 87.3%) - we are not hiding that. Three things to know: it is a higher-level tool that returns synthesized, aggregated results rather than the ~5 raw results a search API returns, which flatters it at single-shot; it shares a vendor with the grader (both Anthropic/Claude), a stated conflict, though the leak check passed; and most importantly it is available only inside Claude/Anthropic agents. If you build on GPT, Llama, Gemini, or your own stack you cannot use it at all - you need a search API like Sofya. Even on Claude, Sofya wins the agentic (real-world) case.

Compare within the harness only. Our agentic Tavily score (93%) matches Tavily's own published 93.3%, a sanity check that the rig is sound. We do not place our numbers next to a competitor's differently-measured figure and call it a win.

Latency is Sofya's own wall-clock (20 varied live queries per depth, June 2026). Competitor latency is not shown - MCP transport overhead is not cleanly subtractable.