BM25 vs vector search: which one should you use?

The debate between BM25 and vector search is one of the most common questions in search engineering. The short answer: neither is universally better. The more useful answer: use both, fused with Reciprocal Rank Fusion (RRF).

Here’s why — and when to weight each signal more heavily.

What is BM25?

BM25 (Best Matching 25) is the dominant term-weighting algorithm in full-text search, used by Elasticsearch, Solr, Lucene, and most search engines by default. It’s an evolution of TF-IDF that addresses two key problems:

1. Term frequency saturation: A word appearing 100 times in a document shouldn’t score 100x higher than a word appearing once. BM25 saturates term frequency using a tunable parameter k1.

2. Document length normalization: A long document has more words, so it’s more likely to contain any term. BM25 normalizes by document length using parameter b.

The scoring formula is:

BM25(D, Q) = Σ IDF(qi) × (tf(qi, D) × (k1 + 1)) / (tf(qi, D) + k1 × (1 - b + b × |D|/avgdl))

In practice: BM25 is fast, explainable, and works without any neural network.

What is vector search?

Vector search represents documents and queries as dense vectors (embeddings) in a high-dimensional space, then retrieves documents whose vectors are closest to the query vector. The distance is typically cosine similarity or dot product.

The critical property: embeddings capture semantic meaning. Two different phrasings of the same concept will produce similar vectors — even if they share no words.

Where BM25 wins

Exact keyword matching. Product codes, part numbers, names, acronyms, and other structured identifiers must match exactly. “SKU-48291-B” should return documents containing exactly that string. BM25 handles this perfectly; vector search may not.

Short queries. A one-word query like “python” is highly ambiguous. BM25 returns documents containing the word “python” — which is probably what the user wants. Vector search may drift toward semantically related concepts that aren’t actually relevant.

Highly technical corpora. Legal citations, regulatory text, and scientific notation rely on exact terminology. BM25 respects this; embedding models may smooth over important distinctions.

Explainability. BM25 scores are interpretable: you can explain why a document ranked highly by pointing to specific matching terms and their frequencies. Vector scores are opaque — “0.87 cosine similarity” doesn’t explain itself.

Computational cost. BM25 is dramatically cheaper to compute than neural embedding. For very high-volume, low-latency workloads where every millisecond counts, BM25 has an advantage.

Where vector search wins

Semantic matching. “How do I cancel my subscription?” should return documents about “account termination,” “plan downgrade,” and “unsubscribe” — even without any shared keywords. Only vector search handles this.

Natural language queries. Users don’t write Boolean queries. They ask questions in natural language. Vector search maps that natural language to a point in the same semantic space as your documents.

Cross-language retrieval. With multilingual embedding models, an English query can retrieve documents in French, German, or Japanese — same vector space, no translation layer required.

Long-tail queries. BM25 fails silently on queries it doesn’t match. Vector search always returns a result — the nearest neighbour is always defined, even for unusual queries.

Zero-shot domain generalization. Pre-trained embedding models generalize surprisingly well to new domains without fine-tuning. BM25 requires no training but is also purely term-based — it can’t generalize beyond the terms it’s seen.

The empirical reality: hybrid wins

Across most retrieval benchmarks, hybrid search (BM25 + vector) outperforms either approach alone. The BEIR benchmark, which spans 18 heterogeneous retrieval tasks, consistently shows that fusion methods outperform individual retrievers.

Why? BM25 and vector search make different errors. When one system returns a false positive, the other often doesn’t. RRF fusion leverages their complementary strengths.

The fusion formula with Reciprocal Rank Fusion:

RRF_score(d) = Σ 1 / (k + rank_i(d))

Where k=60 is a constant that smooths out the effect of high ranks. Documents that appear in both result lists rank higher than documents that appear in only one.

When to weight vector more heavily (higher alpha)

• Natural language questions and conversational queries
• Queries across a multilingual corpus
• Concept-level retrieval (not keyword matching)
• RAG retrieval where semantic context matters more than exact terms
• Long-form queries

When to weight BM25 more heavily (lower alpha)

• Corpora with important structured identifiers (codes, names, SKUs)
• Queries that are short, exact, or technical
• Legal and compliance retrieval where exact term matching matters
• Scenarios where explainability of results is required

Practical tuning

In ManyVector, the alpha parameter controls the weighting:

# Pure vector
results = ns.query(vector=q_vec, search_mode="hybrid", alpha=1.0, top_k=10)

# Balanced
results = ns.query(vector=q_vec, query_text=q_text, search_mode="hybrid", alpha=0.5, top_k=10)

# BM25-heavy
results = ns.query(vector=q_vec, query_text=q_text, search_mode="hybrid", alpha=0.3, top_k=10)

To find the optimal alpha for your corpus, build an evaluation set and sweep alpha from 0.1 to 0.9 in 0.1 increments. Measure recall@10 or NDCG@5. In our experience, alpha=0.65-0.75 wins for most RAG workloads; alpha=0.35-0.45 wins for technical search with structured identifiers.

Summary

Criterion	BM25	Vector	Hybrid (RRF)
Semantic matching	Poor	Excellent	Excellent
Exact keyword matching	Excellent	Variable	Excellent
Natural language queries	Variable	Excellent	Excellent
Cross-language retrieval	No	Yes (multilingual)	Yes
Explainability	High	Low	Medium
Computational cost	Low	Medium	Medium
Overall recall@10	Baseline	+5-15%	+10-25%

For production workloads, start with hybrid search at alpha=0.7. Tune from there based on your eval set results.