Vector search engineering
Why storing vectors in S3 is 10x cheaper than traditional vector databases
A detailed cost breakdown comparing Pinecone, Weaviate, and Qdrant against object-storage-native vector search. The math is uncomfortable for the incumbents.
The anatomy of a RAG pipeline
A detailed look at every component in a production retrieval-augmented generation pipeline — chunking, embedding, retrieval, re-ranking, and generation — with practical tradeoffs at each stage.
HNSW explained: the algorithm powering sub-10ms vector search
A ground-up explanation of Hierarchical Navigable Small World graphs — why they work, how they achieve 90%+ recall at sub-10ms latency, and what the tuning parameters actually do.
Understanding cosine similarity vs dot product in vector search
The practical difference between cosine similarity and dot product for vector search — when they're equivalent, when they diverge, and how to choose the right metric for your embedding model.
BM25 vs vector search: which one should you use?
A practical comparison of BM25 and vector search — when each excels, where they fail, and why hybrid search (RRF) outperforms either approach alone for most production workloads.
Copy-on-write namespaces: Git for your vector database
How copy-on-write branching lets you fork a 100M-vector namespace in milliseconds, run embedding model evals without duplicating data, and roll back cleanly when experiments go wrong.
What is a vector embedding? A practical guide
Vector embeddings explained from first principles: what they are, how they're generated, why they enable semantic search, and how to choose the right embedding model for your workload.
Hybrid search with RRF: why combining vector and BM25 beats either alone
A practical guide to Reciprocal Rank Fusion — the algorithm that combines vector and keyword search scores. With benchmark data showing when hybrid consistently outperforms pure vector retrieval.
Building production RAG pipelines on object storage
A complete architecture guide for RAG pipelines that use object storage as the retrieval layer — covering chunking, embedding, indexing, retrieval, and operational concerns at production scale.
The hidden cost of in-memory vector databases
RAM-first vector databases look affordable on the pricing page. Here's what the pricing page doesn't show: idle RAM tax, replication multipliers, scaling cliffs, and the egress bills that compound over time.
Migrating embedding models with zero downtime using namespace branching
A step-by-step playbook for switching embedding models in production — from evaluation to rollout to rollback — without taking your RAG pipeline offline or duplicating 200GB of vectors.
Metadata filtering in vector search: pre-filter vs. post-filter and why it matters
The difference between filtering before and after vector search is not just semantic — it fundamentally changes recall, latency, and correctness. Here's when to use each and why HNSW pre-filtering is harder than it looks.
Scaling to a billion vectors: lessons from building on object storage
What breaks at 1B vectors that works fine at 1M — memory management, HNSW graph traversal time, compaction, and multi-namespace serving. A technical deep-dive on the engineering challenges of billion-scale vector search.