Metadata filtering in vector search: pre-filter vs. post-filter and why it matters

Every production vector search workload eventually needs metadata filtering. “Give me the 10 most semantically similar documents to this query — but only from the ‘legal’ department, created after 2025-01-01, and not marked as archived.”

The vector search part is solved. The filter part is where things get subtle.

There are two fundamentally different approaches to combining metadata filters with vector search: pre-filtering and post-filtering. Choosing the wrong one produces either incorrect results (wrong documents returned) or poor recall (correct documents missed). The right choice depends on filter selectivity, corpus size, and your latency requirements.

What the terms mean

Post-filtering: Run vector search to get top-K candidates, then apply the metadata filter to the candidates, return whatever passes.

Pre-filtering: Apply the metadata filter to the full corpus first to get a filtered candidate set, then run vector search within that filtered set.

Both sound reasonable. Here’s why each fails under different conditions.

Why post-filtering fails

Suppose you have 1 million documents. You want the top 10 most similar to your query, filtered to documents from the ‘legal’ department.

Only 1,000 of your 1M documents are from the legal department (0.1% selectivity).

Post-filtering steps:

1. HNSW search returns top-100 most similar documents globally.
2. Apply filter: keep only ‘legal’ documents from the top-100.
3. Return result.

The problem: with 0.1% of documents being ‘legal’, the odds that any of the top-100 globally similar documents are in the legal department is low. You might get 0 results from a filter that should return 10. Or you get 3 results and claim that’s the top-3 when you haven’t searched the full legal corpus.

Post-filtering produces incorrect results when the filter is highly selective.

Some systems handle this with “over-fetch” — retrieve top-1000 instead of top-100, then filter. But:

• 10x more HNSW traversal → 10x slower
• If filter selectivity is 0.001%, you’d need top-100,000 to reliably get 10 results
• The over-fetch multiplier approaches the full corpus size for highly selective filters

Why pre-filtering is hard

Pre-filtering fixes the correctness problem but creates an implementation challenge: HNSW was designed to search the full graph, not a subgraph.

The naive pre-filter approach:

1. Find all 1,000 legal documents (fast: a metadata index lookup).
2. Build an HNSW subgraph from just those 1,000 documents.
3. Search the subgraph.

Step 2 is the problem. Building an HNSW graph on 1,000 documents takes time — typically 100ms+ for a well-built index. You can’t build a new subgraph on every query.

Approaches to efficient pre-filtering:

Approach 1: Per-value HNSW indexes

Maintain a separate HNSW index for each distinct metadata value. For department = 'legal', there’s a legal sub-index. For department = 'finance', there’s a finance sub-index.

Pros: fast query (just search the relevant sub-index). Cons: exponential index growth for multi-attribute filters. You can’t efficiently pre-build an index for every possible combination of department AND date_after AND NOT archived.

Works for low-cardinality single-attribute filters. Breaks for complex filters.

Approach 2: Flat search within the filtered set

For highly selective filters (small result set), abandon HNSW entirely. Do a flat (brute-force) scan of the filtered candidate set.

1,000 legal documents × 1536 dimensions × cosine similarity = ~1.5M multiply-add operations. At modern CPU throughput, this takes ~0.5ms. Completely acceptable.

Threshold: flat search is competitive with HNSW up to roughly 50,000-100,000 candidates. Above that, HNSW wins.

# ManyVector automatically chooses the search strategy based on filter selectivity
results = ns.query(
    vector=query_embedding,
    top_k=10,
    filters={"department": {"$eq": "legal"}, "created_after": "2025-01-01"},
    # strategy is chosen automatically: flat for small filtered sets, HNSW for large
)

Approach 3: HNSW with conditional traversal

Modified HNSW search that skips nodes not matching the filter during graph traversal. This maintains the HNSW graph structure but marks filtered-out nodes as “invalid” at query time.

The challenge: HNSW traversal relies on the graph structure to navigate efficiently. Skipping nodes can create disconnected subgraphs that the traversal can’t cross, degrading recall significantly for highly selective filters.

This approach works reasonably well for filters that eliminate 50-80% of the corpus (moderate selectivity). It breaks for very high selectivity (99%+).

Approach 4: Two-phase hybrid

ManyVector uses a two-phase approach:

1. Estimate filter selectivity: count how many documents match the filter (fast, using a secondary attribute index).
2. Choose strategy based on selectivity:
   • < 50,000 candidates: flat search (brute force within filtered set)
   • 50K - 500K candidates: HNSW with conditional traversal

   • 500K candidates: full HNSW search with post-filter and over-fetch

This adaptive approach keeps both correctness and latency under control across the full range of filter selectivities.

Range filters and the cardinality problem

Equality filters (department = 'legal') have clear selectivity. Range filters (date > '2025-01-01') have selectivity that depends on your data distribution.

For a corpus with documents evenly distributed over 5 years, date > '2025-01-01' returns ~20% of documents — low selectivity, HNSW-appropriate. For a corpus with 90% of documents from the past 6 months, the same filter returns 90% of documents — high selectivity, needs careful handling.

ManyVector maintains histogram statistics on indexed numeric and date fields. Before executing a query, it estimates range filter selectivity from the histogram and chooses the optimal execution strategy.

OR conditions and set unions

department IN ('legal', 'compliance') is equivalent to department = 'legal' OR department = 'compliance'. This requires unioning two filtered sets.

For HNSW pre-filtering, you’d search two sub-indexes and merge results. For flat search, you union the candidate sets and run a single brute-force scan.

Set unions are handled differently depending on whether the sub-indexes are cached. ManyVector maintains cached bitsets for common filter values, making union operations O(n/64) using bitwise OR on 64-bit integers rather than O(n) list manipulation.

Practical recommendations

Use pre-filtering as the default. Post-filtering is only correct when filters are very low selectivity (more than 30% of corpus passes the filter). For most real workloads, filters are meant to narrow the search significantly, which means high selectivity, which means post-filtering produces incorrect results.

Index the fields you filter on. ManyVector builds secondary B-tree and bitmap indexes on stored metadata fields. Filtering on unindexed fields requires a full corpus scan, which is slow. Always specify which fields you’ll filter on at namespace creation:

ns = client.create_namespace(
    "my-docs",
    indexed_fields=["department", "date", "source", "user_id"],  # build secondary indexes
)

Combine filters with hybrid search. The filter is a hard constraint; the vector and BM25 scores are soft constraints. Apply the filter first, then rank within the filtered set using hybrid scoring:

results = ns.query(
    vector=query_embedding,
    text=query_text,
    top_k=10,
    fusion="rrf",
    filters={
        "department": {"$eq": "legal"},
        "date": {"$gte": "2025-01-01"},
        "archived": {"$eq": False},
    },
)

Monitor filter selectivity in production. Queries with highly selective filters (returning < 1,000 candidates) have different performance characteristics than broad queries. Set up monitoring on the filtered candidate count per query to detect selectivity outliers that might need query optimization.

Summary

Approach	Best for	Correctness	Latency
Post-filter	Filter selectivity > 30%	Good	Good
Flat pre-filter	Small filtered sets (< 50K)	Excellent	Excellent
HNSW conditional	Medium filtered sets (50K-500K)	Good	Good
Adaptive (ManyVector)	Any selectivity	Excellent	Good

The key insight: filter selectivity is not known in advance and varies by query. An adaptive strategy that estimates selectivity at query time and chooses the appropriate execution plan is the correct general solution. Fixed strategies (always pre-filter, always post-filter) will either be slow or incorrect for some non-trivial fraction of production queries.