Dev.to AI πŸ€– Ai πŸ‘ 0 πŸ“– 5 min read

Which search API gives an LLM the most up-to-date answers about recent events?

On 358 questions whose answers change over time, the two Google SERP APIs in the test gave an LLM the most correct answers: 88.5% and 88.3%. Brave followed at 84.9%, then Exa (83.0%), Parallel (82.7%) and Tavily (81.8%).

On 358 questions whose answers change over time, the two Google SERP APIs in the test gave an LLM the most correct answers: 88.5% and 88.3%. Brave followed at 84.9%, then Exa (83.0%), Parallel (82.7%) and Tavily (81.8%). Each API got one search per question, and one model answered from the results alone.

  • Recent events: the two Google SERP APIs tied. Both were ahead of Tavily, Exa and Parallel, and the gap to Brave was not significant.
  • Why: the test did not measure how old each index is. It did show that the AI Overview at the top of Google's results page carries much of the accuracy. On a preliminary run of 243 FreshQA questions, accuracy fell from 90.5% to 80.2% when the AI Overview was removed from the same results.
  • Trade-off: the two most accurate APIs were also the slowest, at about 8 seconds a search. Exa took 0.74 seconds.

Which search API was most accurate on recent events?

The test used FreshQA, a public set of questions whose answers change over time. A third of the 358 questions carry a false premise, and the model has to point it out. Six APIs got the same questions on November 5, 2026.

Search API Correct on FreshQA (358) 95% interval $ per 1,000 searches Median latency
SerpApi 88.5% 84.8-91.4 $7.25 8.20 s
cloro 88.3% 84.5-91.2 $1.92 7.96 s
Brave Search API 84.9% 80.8-88.3 $5.00 1.21 s
Exa 83.0% 78.7-86.5 $7.00 0.74 s
Parallel 82.7% 78.4-86.2 $5.00 2.99 s
Tavily 81.8% 77.5-85.5 $5.00 2.00 s

The price is the list price per 1,000 searches on the plan that covers 100,000 searches a month. SerpApi bills a second search when part of the page loads separately, and that happened on 75% of FreshQA questions. The latency is the median time of the search request alone, without the model.

A paired test compares two APIs on the same questions and counts the questions that only one of them got right. The two Google SERP APIs had no difference. cloro was more accurate than Tavily, Exa and Parallel, each at p of 0.0012 or lower. The gap to Brave was not significant (p = 0.065). Of the pairs reported among Brave, Exa, Parallel and Tavily, none showed a difference on these questions.

Why do some search APIs miss recent events?

The benchmark measures the answers. It did not measure how old each API's index is, so it cannot say how much of the gap comes from index freshness. Two facts from the setup show where the difference is.

What each API returns. The two Google SERP APIs return what Google shows for the query: answer blocks such as the AI Overview, then the organic results with titles, links and snippets. SerpApi ran with no_cache=true. For the other four, the reader got a title, a URL and an extract of text for each result. Tavily calls it content, Exa highlights, Parallel excerpts, and Brave a description with extra snippets.

The AI Overview. Google writes the AI Overview from several top pages, so the answer is often already in one sentence. The organic results only point at the pages that hold it. On a preliminary run of 243 FreshQA questions, the AI Overview was removed from one Google SERP API's saved results, and nothing else changed. Accuracy fell from 90.5% to 80.2%.

So in this test, the answer block at the top of Google's page carried a large part of the accuracy on these questions. For the other four APIs, the reader got no such summary. The test cannot separate that effect from the age of each index.

When is another API the better choice?

The order changes with the question and with what the agent can wait for.

  • Speed. If your agent must answer in under 3 seconds, the two Google SERP APIs are not the right choice today. Exa took 0.74 seconds and Brave 1.21 seconds at the median.
  • Brave on recent events. Brave scored 84.9%, and its gap to one of the two leaders was not significant (p = 0.065). Its median search took 1.21 seconds, against 7.96 seconds.
  • Short factual questions. On 1,000 questions with one stable answer (SimpleQA Verified), Parallel (97.8%), Exa (97.3%) and one of the two Google SERP APIs (96.9%) were within the margin of error of each other.

How was the test run?

Every API got the same setup:

  1. One search per question. The query is the question text, unchanged. There is no query rewrite, no second search and no page fetch.
  2. 10 results in the API's default search mode. Answer features sold as separate options (Tavily include_answer, Exa summary) were off.
  3. The same reader. One model (gpt-5.6-luna) read at most 30,000 characters of results and answered from them alone, or replied "NOT FOUND". For FreshQA, the prompt gives today's date and asks the reader to point out a false premise.
  4. The same grader. One model (gpt-5.6-sol) graded each answer correct, incorrect or not attempted with the unchanged OpenAI simple-evals grader prompt.
  5. The same questions, shuffled with seed 42.

What does this test not show?

  • The fastest-changing questions. The set is the FreshQA test split of April 21, 2026, without the fast-changing questions and without the questions past their review date. So the 358 questions do not test news from the last few hours.
  • Answer keys go stale. On 30 FreshQA questions, most APIs disagree with an answer key that may be out of date. Without those questions, the order of the APIs does not change.
  • An agent. One search measures the search API. An agent that searches many times and opens pages can get a high score from a weaker search API.
  • The model's memory. The reader can answer from what it already knows. The prompt forbids it, and the same reader read every API's results, so the effect is the same for all six.

How do you run the test again?

The harness is one Python file with no packages to install. Set OPENAI_API_KEY and the keys of the APIs you want to compare, then run:

python3 benchmark.py --dataset freshqa --providers cloro,serpapi,tavily,exa,parallel,brave --n 100 --workers 4

Run --n 3 first to find a broken key before you spend the budget. --as-of YYYY-MM-DD sets the date in the reader prompt and the FreshQA filter, so you can read the saved responses of an earlier run again with the same questions.

How this was checked

This test is run by cloro, which sells one of the two Google SERP APIs in it. For that reason, the harness, the request settings for each API and every graded answer are public in the benchmark repo. If a setting is unfair to an API, change it and run it again. Prices were read on November 5, 2026, from each vendor's pricing page.

The figures above come from the web search API benchmark, which also has the results on multi-hop and short factual questions, the cost per correct answer, and the tests of which parts of a search result carry the accuracy.

πŸ“° Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes β€” full credit and traffic to the original publisher.