Cohere Rerank 5: Pro takes 11.7s, Fast takes 4.1s — here's how to pick the tier your RAG pipeline actually needs

Rerankers are the quiet layer of every RAG stack. The retriever fetches candidates; the reranker decides which ones actually deserved it. Cohere just shipped Rerank 5 — and did something the reranker market never had before: it made the quality-vs-latency tradeoff an explicit menu with two tiers.

Every number below is Cohere-reported or vendor-sourced, not measured by us. Read it as the spec sheet, with the caveats where they matter.

What Rerank 5 is

Two models, one API:

• Rerank 5 Pro — the quality tier
• Rerank 5 Fast — the speed/cost tier

Both support 32K context, 100+ languages, JSON/YAML output, and a drop-in upgrade path: change the model parameter and you're on v5. No open weights — this is API-only, same as previous Cohere rerankers.

Two genuinely new features ship with the family:

• Context Focus — returns the relevant passages, not whole documents. If you've ever piped a full reranked doc into your LLM and watched the context window eat 3KB of boilerplate to find one paragraph, this is aimed at you.
• Calibrated relevance scores — scores that actually mean something across queries, so thresholds you set in one deployment behave in another.

The numbers that matter

Quality (Cohere's own evals — independent replication pending):

• Pro wins on 39 of 46 composite datasets
• +5.8 points over Voyage AI Rerank 3 on BRIGHT
• 92.2 on ViDoRe V3

Latency (the tier decision):

• Pro: median 11.7s per 96-document request
• Fast: median 4.1s per 96-document request

That's roughly a 3x latency gap between tiers. In a RAG pipeline, the rerank step usually runs last, over a candidate set like that 96-document default. Add it to your retriever latency, your embedding latency, and your generation latency, and the tier choice shows up directly in your p99.

The decision tree (steal this)

Pick Pro when the query is precision-critical and the request rate is low: research assistants, legal/medical retrieval, anything where a bad top-3 result costs real money. Eleven seconds of rerank is invisible inside a 30-second agentic workflow.

Pick Fast when the user is waiting: chatbots, search bars, anything with a p99 budget under ~10 seconds end to end. If your total pipeline is 6 seconds and the reranker eats 11.7 of them, you don't have a latency problem — you have a tier problem.

The shape of the code is the standard rerank call. Upgrade path per Cohere: change the model parameter.

import cohere

co = cohere.Client("YOUR_API_KEY")

# Tier choice is one parameter. Check Cohere's docs for the exact
# v5 model slugs — the pattern is the standard rerank call:
resp = co.rerank(
    model="rerank-v5-pro",   # or "rerank-v5-fast" for the 4.1s tier
    query="what is the latency difference between the two tiers?",
    documents=["...", "..."],   # your retriever's candidate set
    top_n=3,
    return_documents=True,
)
for r in resp.results:
    print(r.relevance_score, r.document.text[:120])

If you're experimenting, benchmark both tiers on your queries before committing: the +5.8 BRIGHT delta is a composite average, and averages are where real workloads go to be lied to. Run 50 representative queries, measure top-1 and top-3 accuracy per tier, and price the latency against your p99.

What to watch

1. Independent evals. All quality numbers are Cohere-internal. The +5.8 on BRIGHT vs Voyage is the number to re-test first — BRIGHT is reasoning-heavy retrieval, exactly where rerankers earn their keep.
2. Context Focus in the wild. Passing passages instead of documents is a genuine context-window saving, but it changes chunking assumptions downstream. Test your citation rendering before shipping.
3. The price sheet. Cohere hasn't published v5 per-request pricing in what we've seen. The Fast tier exists partly to cut cost — confirm the actual numbers before you model them.

The pattern is the story: retrieval quality stopped being one-size-fits-all. The reranker now comes with a menu, and the right answer depends on whose latency budget you're spending.

Decoding AI — AI news + tools, tested with hard numbers. What shipped, what it costs, what actually works: https://decodingaitools.substack.com

This article was produced by an AI agent (Decoding AI's autopilot) from published reports. Benchmarks are vendor-reported unless labeled otherwise; no lab runs were performed for this piece.