Is there an independent benchmark of web search APIs like Tavily, Exa and Brave?

Yes, three third-party boards test Tavily, Exa and Brave Search API side by side: Openbenchmarks, AIMultiple and Artificial Analysis. None of the three sells a search API, and each keeps the model, the questions and the grader fixed while it changes the search API. They measure different things, and each one keeps part of its question set private.

• Openbenchmarks runs one search per question on 300 company-news lookups, plus agent tasks for hard retrieval (100 tickets) and multi-hop search (45 questions). Its harnesses are MIT-licensed on GitHub, with public question samples on Hugging Face.
• AIMultiple and Artificial Analysis give a fixed AI agent many searches per question. They measure the search API as one tool inside an agent loop.
• Vendor benchmarks from Tavily, Exa, Parallel and cloro often publish more of their code and answers, but the company that runs each one sells an API in the table. Tavily's and Exa's own tables put their own API first.

Which independent benchmarks test Tavily, Exa and Brave?

What Openbenchmarks measured

On the lookup board, every API gets the question text unchanged, one request and at most 10 results. One model (gpt-5.6-terra) reads the titles and snippets and extracts an answer. A second model (claude-opus-5) judges it against answers that people labelled before any vendor was called.

Exa (fast mode) scored 99.3%, Exa instant 97.7%, Brave's LLM context endpoint 94.0%, Brave web search 93.3%, Tavily advanced 93.0% and Tavily basic 87.7%. The board itself flags a problem with these numbers: vendors benchmarked later tend to score higher. Until a re-run, it says the lookup rankings "should not be used to compare vendors."

On the agent boards, Exa deep led hard retrieval at 83.0% when the agent could fetch pages, and Parallel basic led multi-hop search at 46.5% F1 without page fetch.

Its about page says "Nobody pays to be included, ranked or removed." The same page says it sells evals as a service and that many of its partners are customers. Published rankings come from a held-out set whose questions it never publishes.

What AIMultiple measured

A fixed agent (OpenCode with gpt-5.6-luna) answered 120 English questions with each of 24 configurations of Brave, Exa, Firecrawl, Parallel, Perplexity and Tavily. Three judge models graded each answer, and an answer counts as correct when at least two of them agree. Parallel Advanced led at 87.0% (95% interval 78.3 to 94.2) and Exa Fast followed at 82.6% (73.9 to 91.3). The page says these intervals overlap and each configuration ran once per question. It also states that the research "does not mention any subscribers of AIMultiple's benchmarking services."

What Artificial Analysis measured

A fixed agent (gpt-5.6-luna, medium effort) gets web_search and web_fetch tools and up to 25 turns. The index is the equal-weighted mean of three benchmarks: DeepSearchQA F1, BrowseComp accuracy and AA-Omniscience accuracy. The configurations include Tavily basic, Exa lite and auto, and Brave web and LLM context. The methodology page shows no scores and makes no statement about vendor payment. This article did not read the leaderboard, so it gives no score from it.

Which benchmarks are run by search API vendors?

The same APIs can come out in the opposite order. On Tavily's SimpleQA run, Exa scored 71.24% and Tavily 93.3%. On SimpleQA Verified in the six-API benchmark, Exa scored 97.3% and Tavily 91.8%. The question sets, the reader models and the request settings differ, so neither result transfers to the other setup.

What about blog posts that call themselves benchmarks?

• "I benchmarked 4 web search APIs for AI agents" on dev.to tested You.com, Tavily, Exa and Parallel on 30 agent-style queries, 3 runs each, on October 8, 2026. As a share of the maximum score, Parallel scored 67%, You.com 63%, Exa 53% and Tavily 51%. The harness, answer keys and grades are MIT-licensed on GitHub. The author calls it "a focused test, not a leaderboard." Brave is not in it.
• "Benchmarking LLM Search APIs: Tavily vs Web Search Plus vs OpenAI Web Search" on dev.to is one question, written by the team that built Web Search Plus. It compares the cost of one answer. It has no Exa or Brave result.
• Braintrust and Vellum publish roundups of web search APIs. Braintrust's own numbers come from a study of You.com against built-in model search. Vellum quotes Openbenchmarks and AIMultiple.

How does the open six-API benchmark work?

1. One search per question. The query is the question text, with no rewrite, no second search and no page fetch.
2. 10 results in each API's default mode. Answer features sold as extra options (Tavily include_answer, Exa summary) are off.
3. One reader. gpt-5.6-luna reads at most 30,000 characters of results and answers from them only, or says "NOT FOUND".
4. One grader. gpt-5.6-sol grades each answer with the unchanged OpenAI simple-evals grader prompt.
5. The same questions, shuffled with seed 42, sent to every API question by question, 4 requests at a time, from one machine. The runs were on November 4, 5, 10 and 11, 2026.

No API is first on all three sets. Parallel, Exa and cloro are within the margin of error on SimpleQA Verified, SerpApi and cloro tie on FreshQA, and cloro leads FRAMES. The limits are in the README: one search measures the search API and not an agent, 193 of the 1,000 SimpleQA questions had a result that names the dataset, and 30 FreshQA answer keys may be out of date. Without those questions, the order does not change.

How do you rerun it?

The harness is one Python file with no packages to install.

1. Print the published FRAMES table from the saved answers. This needs no API key:

   python3 benchmark.py --summary --out runs/frames-2026-11

1. Set OPENAI_API_KEY and the keys of the APIs you test: TAVILY_API_KEY, EXA_API_KEY, BRAVE_API_KEY.
2. Run 3 questions first to find a broken key, then 100:

   python3 benchmark.py --dataset frames --providers tavily,exa,brave --n 3 --workers 4
   python3 benchmark.py --dataset frames --providers tavily,exa,brave --n 100 --workers 4

1. Change --dataset to simpleqa or freshqa for the other sets. --dataset obnews loads the 100 public Openbenchmarks company-news questions into the same harness.

The harness has no option for a question file of your own, so pick the set that is most like your agent's questions.

How should you read any web search API benchmark?

1. Who runs it? Check whether the publisher sells an API in the table, and whether it sells services to the vendors.
2. One search or an agent loop? One search measures the results page. An agent with 25 calls can make a weak API look strong.
3. Which mode of each API? On the Openbenchmarks lookups, Tavily advanced scored 93.0% and Tavily basic 87.7%. A score belongs to the mode that was tested.
4. Can you see the answers? With a public question set and the graded answers, you can check a result yourself. A held-out set protects the board against tuning, but you cannot check it.
5. How many questions, and when? With 120 questions, AIMultiple's top intervals overlap. Prices, indexes and modes change from month to month.

Does any benchmark compare Serper or SerpApi with Tavily, Exa and Brave?

None of the three independent boards tests Serper or SerpApi (two Google SERP APIs). Openbenchmarks has one Google endpoint, a RapidAPI listing, which scored 96.0% on lookups and an F1 of 0.4 on multi-hop search without page fetch, the lowest on that board. Google results appear with Tavily's harness through Serper and with the six-API benchmark through SerpApi and cloro.

How this was checked

This article is published by cloro, which sells one of the six APIs in its own benchmark and ran that benchmark. cloro is not on the Openbenchmarks, AIMultiple or Artificial Analysis boards. Every third-party figure was read on its own page or repo on November 15, 2026, from the links above. The six-API figures are copied from the benchmark README, not recomputed, and the FRAMES column was printed again with the --summary command above. The cost, latency and head-to-head tests are in the benchmark write-up.