Hybrid Search for RAG: BM25 + Vector Combined (2026 Guide)

Pure vector search misses what pure keyword search catches, and vice versa. Hybrid retrieval runs both and merges the results, which closes the gap on queries where either approach alone fails.

This guide covers when hybrid helps, the two main merge strategies (reciprocal rank fusion and weighted score), and how to implement hybrid retrieval in Weaviate, Pinecone, Qdrant, and plain Python. Expect 10 to 25% lift on top-5 recall over pure vector search on realistic workloads.

Why hybrid search exists

Vector search is strong at semantic similarity: “cancel my plan” matches a chunk talking about “subscription termination” even without word overlap. It is weak at exact-token lookup. A query like “error code E-4042” or “section 11.3.c” or “voyage-code-2 model” often fails to retrieve the right chunk because the specific identifier does not land close in embedding space to variations of itself.

BM25 keyword search is the inverse: strong at exact and near-exact token matches, bad at semantic generalization. The two together cover both failure modes.

Four query patterns where hybrid beats pure vector by a noticeable margin:

• Named entities and identifiers. Product SKUs, invoice numbers, order IDs, legal citations, API endpoint names.
• Acronyms the model has not seen often. Industry-specific abbreviations where the embedding model lacks training examples.
• Rare technical terms. Function names, variable names in source code, chemical compounds, drug names.
• Phrase-level queries. “Right to be forgotten” needs that exact phrasing retrieved, not a loose paraphrase.

Pure vector search handles the other 80% of queries fine. Hybrid is specifically for the long tail.

How BM25 works in 30 seconds

BM25 scores a chunk against a query by three signals: term frequency (how often each query word appears in the chunk), inverse document frequency (how rare each word is across the whole corpus), and chunk length normalization (penalize chunks that match just because they are long).

A chunk that contains “voyage-code-2” scores high if the term is rare across the corpus and appears in the chunk at least once. A chunk that contains “the” and nothing else interesting scores low because “the” has terrible IDF.

No training is required. BM25 is the baseline that worked for search engines for 25 years before vector search arrived. In 2026 it is still the right companion to dense retrieval.

Two ways to merge: RRF vs weighted score

Reciprocal rank fusion (RRF) ignores raw scores and merges on rank. If a chunk is ranked 2 by vector and 5 by BM25, its RRF score is 1/(60+2) + 1/(60+5), where 60 is a smoothing constant. Higher is better. Chunks that rank high on either retriever bubble to the top; chunks that rank high on both bubble higher.

Weighted score combines the two retrievers’ scores with a tunable alpha: final_score = alpha * vector_score + (1 - alpha) * bm25_score. Alpha of 0 is pure BM25, 1 is pure vector. Most production systems settle on 0.4 to 0.6.

RRF is parameter-free, which is a real advantage. You do not have to re-tune the alpha when documents change. Weighted score lets you push the system toward one retriever when you know your query distribution leans a specific way.

Hybrid search in Weaviate

Weaviate has had native hybrid search since v1.17. One query, both retrievers, built-in RRF or alpha-weighted merge.

import weaviate
from weaviate.classes.query import HybridFusion

client = weaviate.connect_to_local()
col = client.collections.get("Docs")

results = col.query.hybrid(
    query="error code E-4042 troubleshooting",
    alpha=0.5,
    fusion_type=HybridFusion.RELATIVE_SCORE,
    limit=10,
)

for obj in results.objects:
    print(f"[{obj.metadata.score:.3f}] {obj.properties['title']}")

Alpha of 0.5 gives equal weight to vector and BM25. HybridFusion.RELATIVE_SCORE normalizes both score ranges to 0-1 before combining. For RRF-style merging, use HybridFusion.RANKED.

Hybrid search in Pinecone

Pinecone’s hybrid uses sparse-dense vectors. You generate sparse vectors at index time using BM25 or SPLADE, then query with both the dense and sparse components.

from pinecone_text.sparse import BM25Encoder
from pinecone import Pinecone

bm25 = BM25Encoder.default()
bm25.fit(corpus_texts)

pc = Pinecone(api_key=os.environ["PINECONE_API_KEY"])
index = pc.Index("my_hybrid_index")

# Index time
for chunk in chunks:
    index.upsert(vectors=[{
        "id": chunk["id"],
        "values": dense_embed(chunk["text"]),
        "sparse_values": bm25.encode_documents(chunk["text"]),
        "metadata": chunk["metadata"],
    }])

# Query time
query = "error code E-4042 troubleshooting"
results = index.query(
    vector=dense_embed(query),
    sparse_vector=bm25.encode_queries(query),
    top_k=10,
    include_metadata=True,
)

Pinecone’s hybrid is more work to set up because you own the BM25 encoder, but it scales to billions of vectors. For large-scale hybrid, Pinecone is production-grade.

Hybrid search in Qdrant

Qdrant supports hybrid via its multi-vector feature. Store a dense vector and a sparse vector per chunk, query both, fuse with RRF.

from qdrant_client import QdrantClient
from qdrant_client.models import Prefetch, Fusion, FusionQuery, SparseVector

qc = QdrantClient("localhost", port=6333)

results = qc.query_points(
    collection_name="docs",
    prefetch=[
        Prefetch(query=dense_vec, using="text-dense", limit=20),
        Prefetch(query=SparseVector(**sparse_vec), using="text-sparse", limit=20),
    ],
    query=FusionQuery(fusion=Fusion.RRF),
    limit=10,
)

Qdrant’s prefetch pattern is clean. Each prefetch is a sub-query; the final query fuses them. You can prefetch 3 or 4 retrievers (dense, BM25, ColBERT, etc.) and fuse them all in one call.

Rolling your own hybrid in Chroma

Chroma does not have hybrid search in 2026. For teams on Chroma, run BM25 separately and merge in Python. The rank_bm25 package handles the sparse side.

from rank_bm25 import BM25Okapi

tokenized_corpus = [chunk["text"].split() for chunk in all_chunks]
bm25 = BM25Okapi(tokenized_corpus)

def hybrid_retrieve(query, k=10):
    vector_results = collection.query(query_texts=[query], n_results=20)
    vector_ranked = [
        (doc_id, 1 - dist, i)
        for i, (doc_id, dist) in enumerate(
            zip(vector_results["ids"][0], vector_results["distances"][0])
        )
    ]

    bm25_scores = bm25.get_scores(query.split())
    top_bm25_idx = sorted(
        range(len(bm25_scores)), key=lambda i: -bm25_scores[i]
    )[:20]
    bm25_ranked = [
        (all_chunks[i]["id"], bm25_scores[i], rank)
        for rank, i in enumerate(top_bm25_idx)
    ]

    rrf_scores = {}
    for doc_id, _, rank in vector_ranked:
        rrf_scores[doc_id] = rrf_scores.get(doc_id, 0) + 1 / (60 + rank)
    for doc_id, _, rank in bm25_ranked:
        rrf_scores[doc_id] = rrf_scores.get(doc_id, 0) + 1 / (60 + rank)

    return sorted(rrf_scores.items(), key=lambda x: -x[1])[:k]

Fifty lines of Python gives you production-quality hybrid search. The constant 60 in 1 / (60 + rank) is the standard RRF smoothing; it damps the top-rank advantage so a #1 ranked chunk is not infinitely more valuable than a #2.

Measuring the lift

Build a 50-100 query ground-truth set. For each query, label which chunk IDs are the correct answer. Run three retrievers:

• Pure vector, top-5 recall
• Pure BM25, top-5 recall
• Hybrid RRF, top-5 recall

Typical results on realistic workloads:

• Pure vector: 65-80% top-5 recall
• Pure BM25: 50-70% top-5 recall
• Hybrid: 75-90% top-5 recall

Hybrid consistently beats either alone. The lift is largest on queries with named entities, identifiers, and rare technical terms. On generic semantic queries, hybrid and pure vector tie.

When hybrid is not worth it

Two cases where staying with pure vector is defensible:

Pure conversational queries. A customer support bot where users ask “how do I cancel” or “where is my refund” rarely benefits from BM25. The queries are paraphrase-heavy, which is where vectors win.

Short query, long chunk. If queries are 3-5 words and chunks are 500+ tokens, BM25 scoring is noisy. The signal is weak and vector search carries the load alone.

For most other RAG workloads, hybrid is a free upgrade. Twenty-four lines of code for a 15% recall lift is a good trade.

Beyond BM25: SPLADE and ColBERT

Two newer sparse retrievers worth knowing about in 2026:

SPLADE. Learned sparse encoder. Expands queries and documents with related terms using an LLM, then does BM25-style scoring on the expanded forms. 2-5% better than BM25 on most tasks, but more expensive at index time.

ColBERT. Late-interaction retrieval. Each chunk gets a vector per token (not a single vector per chunk). Scoring is more expensive, but accuracy is noticeably higher on long-form content. ColBERTv2 and bge-m3 (which supports ColBERT-style multi-vector mode) are the main production options.

For most teams, BM25 + dense retrieval + a reranker is the right stopping point. SPLADE and ColBERT are the next step when retrieval quality still matters more than complexity cost.

Frequently asked questions

Hybrid is a free upgrade on most RAG systems. Pick RRF to skip tuning. Add it before you reach for more complex optimizations like reranking or query rewriting. The gains compound, but hybrid search is where you get the easiest wins.

Joken E. Villanueva

Founder & Lead Developer at PIES IT Solution

Founder of PIES Information Technology Solutions, a software company building production-grade applications for institutions across the Philippines. Over 8 years of hands-on full-stack development experience, currently leading the development of ClinicAI, an AI-powered clinic management platform.

Expertise: PHP · MySQL · JavaScript · AI Integration · SaaS Architecture · VB.NET · Database Design · Capstone Documentation · Java  · View all posts by Joken E. Villanueva →