Hybrid Search for RAG: BM25 + Vectors in 50 Lines of Python

Hybrid search runs keyword search and vector search in parallel, then merges the two ranked lists into one, usually with Reciprocal Rank Fusion. It matters because the two methods fail in opposite ways: vector search misses product codes and proper nouns, while keyword search misses rephrased questions. The fusion recovers both signals for a few milliseconds.

Key takeaways

• BM25 catches exact terms (codes, names, identifiers) that embeddings routinely miss; dense vectors catch paraphrases that keyword search misses. Their failures barely overlap.

• You cannot add a BM25 score and a cosine similarity together: one is unbounded and corpus-dependent, the other is bounded. Fuse ranks, not scores.

• Reciprocal Rank Fusion, RRF(d) = sum of 1/(k + rank), needs no score normalisation and takes about 15 lines of Python, with k = 60 by convention.

• Most vector databases already ship hybrid search natively, but building it by hand shows you exactly what is happening under the hood.

• The gains are real but corpus-dependent: measure recall on your own documents before assuming hybrid wins.

Why pure vector search is not enough

Embeddings are trained to capture meaning, and that training makes them careless about surface form. A dense retriever asked about ticket KB-2024-7831 will happily return three articles about ticket-tracking systems, none of which contains the string KB-2024-7831, while a BM25 index returns the exact article in position one. That example comes from a 2026 implementation write-up on dev.to, and it matches what production RAG teams report: the single most common retrieval bug is an exact identifier that the vector search never finds.

BM25 has the mirror problem. It scores documents by token overlap, term frequency and document length, so "how do I reset my password" matches a page titled "Account recovery steps" only if the words line up. A 2026 practitioner post on Medium put it well: dense retrieval catches paraphrases and conceptual matches, while sparse retrieval catches the literal strings that dense retrieval was never built to represent. The two families fail on almost disjoint query types, which is precisely why combining them works.

Why can't you just add the two scores together?

This is the obvious first idea, and it breaks on the arithmetic. BM25 scores are unbounded positive numbers tied to your corpus statistics: in my demo below, the winning BM25 score is 7.289 and the losers score 0.000. Cosine similarities live in the range [-1, 1] and cluster narrowly in practice: the same demo's winning dense score is 0.584. A BM25 score of 7.289 means nothing without knowing your corpus's term-frequency distribution, so any min-max or z-score normalisation you apply before adding is fragile and shifts whenever your corpus changes.

The elegant escape is to ignore the scores entirely and keep only the ranks. That is Reciprocal Rank Fusion (RRF), introduced by Cormack, Clarke and Büttcher in a 2009 SIGIR paper. For each document, you add up 1/(k + rank) across every retriever, with k = 60 by long-standing convention. A document ranked first by both methods scores 1/61 + 1/61 = 0.0328, the maximum. A document ranked first by one method and 50th by the other scores 1/61 + 1/110 = 0.0255, still respectable. Ranks are on the same scale by definition, so the fusion never needs calibration. That property is why RRF has survived 17 years of retrieval research essentially unchanged.

The 50-line implementation

Before reaching for a vector database's native hybrid mode, build the whole thing by hand once. The code below implements BM25 from scratch (Robertson and Zaragoza's formulation, k1 = 1.5, b = 0.75), a TF-IDF cosine stand-in for the dense side, and RRF fusion. In production you would swap the TF-IDF vectors for real embeddings; the fusion code is byte-for-byte identical, because RRF only ever sees rank orders.

import math, re
from collections import Counter

def tokenize(text):
return re.findall(r"[a-z0-9]+", text.lower())

class Corpus:
K1, B = 1.5, 0.75

def __init__(self, docs):
    self.docs = docs
    self.toks = [tokenize(d) for d in docs]
    self.doc_len = [len(t) for t in self.toks]
    self.avgdl = sum(self.doc_len) / len(self.doc_len)
    self.df = Counter(t for toks in self.toks for t in set(toks))
    self.vocab = sorted(self.df)
    self.N = len(docs)

def bm25(self, query):                      # sparse side
    qtoks = tokenize(query)
    scores = []
    for i, toks in enumerate(self.toks):
        tf = Counter(toks)
        s = 0.0
        for t in qtoks:
            if t not in tf:
                continue
            idf = math.log(1 + (self.N - self.df[t] + 0.5)
                             / (self.df[t] + 0.5))
            s += idf * (tf[t] * (self.K1 + 1)) / (
                tf[t] + self.K1 * (1 - self.B
                     + self.B * self.doc_len[i] / self.avgdl))
        scores.append(s)
    return scores

def dense(self, query):                     # TF-IDF cosine stand-in
    qv = self._tfidf(tokenize(query))
    return [self._cos(qv, self._tfidf(t)) for t in self.toks]

def _tfidf(self, toks):
    tf = Counter(toks)
    return [tf[t] * math.log(1 + self.N / self.df[t])
            if t in self.df else 0.0 for t in self.vocab]

@staticmethod
def _cos(a, b):
    na = math.sqrt(sum(x*x for x in a))
    nb = math.sqrt(sum(x*x for x in b))
    return sum(x*y for x, y in zip(a, b)) / (na*nb) if na and nb else 0.0

def rrf(ranked_lists, k=60): # the fusion
scores = {}
for ranked in ranked_lists: # each list: best doc first
for rank, doc_id in enumerate(ranked, start=1):
scores[doc_id] = scores.get(doc_id, 0.0) + 1.0 / (k + rank)
return sorted(scores, key=scores.get, reverse=True)

def ranks(scores):
return sorted(range(len(scores)),
key=lambda i: scores[i], reverse=True)

def hybrid_search(corpus, query, top_n=3):
fused = rrf([ranks(corpus.bm25(query)),
ranks(corpus.dense(query))], k=60)
return fused[:top_n]

What it actually finds

Running this against an 8-document toy corpus (database docs, one with the identifier KB-2024-7831, one refund policy, one team-lunch notice) produces real output, not a sketch:

BM25 for: "KB-2024-7831 connection failure"
#1 doc3 (score=7.289) KB-2024-7831: connection pool exhaustion on the EU Frankfurt rep...
#2 doc0 (score=0.000) PostgreSQL full text search uses tsvector and tsquery...

dense (TF-IDF proxy) for: "KB-2024-7831 connection failure"
#1 doc3 (score=0.584) KB-2024-7831: connection pool exhaustion on the EU Frankfurt rep...
#2 doc0 (score=0.000) PostgreSQL full text search uses tsvector and tsquery...

-- RRF fused --
#1 doc3 KB-2024-7831: connection pool exhaustion on the EU Frankfurt rep...

Two things to notice. First, the identifier query is exactly the case where dense retrieval fails in production; my TF-IDF stand-in still finds it here only because the identifier tokens overlap, and a real embedding model would be far more likely to miss it. Second, look at the raw scores: 7.289 versus 0.584. Had you added them, the BM25 side would have swallowed the dense side whole. RRF's rank-only math puts doc3 first without ever comparing those incompatible numbers.

Does hybrid actually beat dense-only? What measurements exist

Honest answer first: it depends on your corpus, and you should measure. The published numbers are encouraging but small-scale. One practitioner ran 12 questions against a company earnings transcript in October 2026 and reported Hit@3 rising from 9/12 (BM25) and 10/12 (dense) to 12/12 (hybrid), with mean reciprocal rank moving from 0.66 and 0.73 to 0.86. Another October 2026 write-up headlined "30% better recall" from combining BM25 with vectors, and a hybrid RAG guide reports roughly 91% recall@10 with BM25 plus RRF and no score-normalisation tricks. Treat these as existence proofs, not guarantees: they are single-corpus experiments by individual authors, and the 91% figure comes from a vendor-adjacent guide whose exact test setup is not fully disclosed.

The pattern across them is consistent though: hybrid never loses to either side alone by much, and it wins clearly whenever the query set mixes identifiers with paraphrases. That is the realistic expectation to carry into your own evaluation.

What hybrid search costs you

The cost is smaller than most engineers assume. Running both retrievers in parallel adds roughly 6 milliseconds at p50 over dense-only search, according to the 2026 dev.to implementation write-up. A cross-encoder reranker on the top 50 fused candidates adds another 50 to 200 milliseconds depending on the model, per the AI Workflow Lab hybrid guide, but the reranker is optional: RRF alone is already the production default in many systems.

The real costs are operational. You maintain two indexes instead of one, you pick a candidate depth per side (top 50 to 100 each is the usual range), and you choose k for RRF. The honest truth is that nobody knows the optimal k for your corpus without trying: 60 is a convention that works well across domains, not a tuned constant. If you would rather tune than fuse by rank, weighted score fusion is the alternative: Weaviate weights dense at 0.75 and Pinecone at 0.5 in their defaults, per their documentation. RRF's advantage is that it needs no such calibration step.

When should you skip hybrid search?

Not every RAG system needs it. Skip the hand-rolled version when your vector database already does it natively: Qdrant, Weaviate, Elasticsearch, OpenSearch and Milvus all support BM25 or sparse vectors alongside dense vectors in the same collection, with native RRF fusion. That is the right call for most teams.

Skip hybrid entirely when your corpus is small and homogeneous enough that one retriever covers it, or when your queries are genuinely uniform, for example an internal FAQ where every question is phrased like the answers. And if Postgres is already your database, note that pgvector plus tsvector gives you dense and keyword search in one service, as the dev.to write-up demonstrates, so hybrid does not have to mean a second piece of infrastructure.

The production version of this pipeline

The shape that has settled in production RAG systems by 2026 is four stages: generate candidates from both a sparse and a dense index in parallel, fuse with RRF, optionally rerank the top 50 with a cross-encoder, then hand the winners to the LLM. LangChain's EnsembleRetriever follows exactly this pattern, using RRF under the hood with k = 60 hardcoded and a weights parameter that scales each retriever's contribution before fusion.

The code in this article is the whole of stage two and a from-scratch stage one. Once you have seen RRF work on your own corpus, the vector database's "hybrid mode" stops being magic and becomes a config option you can reason about. That understanding is worth more than the 50 lines.

Originally published on LogicStack. Run the code examples right in your browser with our free interactive Python and SQL playgrounds — no setup, no sign-up.