Migrating embedding models with zero downtime using namespace branching

Embedding model migrations are one of the most feared operations in a production RAG system. Switching from text-embedding-3-small to text-embedding-3-large — or from OpenAI to Cohere, or from a general model to a fine-tuned domain-specific one — requires re-embedding your entire corpus.

For a corpus of 10M documents, that means:

• Re-running 10M embedding API calls
• Storing 10M new vectors
• Validating that the new model actually improves retrieval quality
• Cutting over production traffic without downtime

The naive approach: spin up a new empty namespace, re-embed everything into it, test it, then switch. This requires 2x storage during the migration and creates a period where your new namespace is out of sync with production writes.

The better approach: namespace branching.

The migration playbook

Phase 1: Baseline measurement

Before touching anything, measure your current retrieval quality. Without a baseline, you can’t know whether the new model is actually better.

Build an eval set: 100-500 (query, relevant_document_ids) pairs. These should reflect your real query distribution. Include:

• Queries where you know the answer exists in the corpus
• Queries that your users actually ask (sample from your logs)
• Adversarial queries that should return no results

def measure_retrieval_quality(namespace_name: str, eval_set: list[dict]) -> dict:
    ns = client.namespace(namespace_name)
    hits = 0
    
    for item in eval_set:
        query_embedding = embed(item["query"])
        results = ns.query(vector=query_embedding, top_k=10)
        retrieved_ids = {r.id for r in results}
        
        if any(doc_id in retrieved_ids for doc_id in item["relevant_ids"]):
            hits += 1
    
    return {
        "namespace": namespace_name,
        "recall_at_10": hits / len(eval_set),
        "n_queries": len(eval_set),
    }

# Measure before touching anything
baseline = measure_retrieval_quality("my-docs", eval_set)
print(f"Baseline recall@10: {baseline['recall_at_10']:.3f}")

Phase 2: Fork the namespace

Create a branch from production. This is instantaneous — no data is copied:

# Fork production namespace — completes in milliseconds regardless of corpus size
eval_ns = client.namespace("my-docs").fork("my-docs-new-model-eval")
print(f"Forked namespace created. Vectors inherited: {eval_ns.vector_count}")

The forked namespace starts as an exact copy of production from the query perspective. All existing vectors are shared (copy-on-write). New writes go to the fork only.

Phase 3: Evaluate on a sample first

Before re-embedding the entire corpus, test on a 1-5% sample. This validates the hypothesis quickly and cheaply:

import random

all_docs = fetch_all_document_ids()  # from your document store
sample_ids = random.sample(all_docs, k=int(len(all_docs) * 0.02))  # 2% sample

# Re-embed the sample with the new model
for doc_id in sample_ids:
    doc = fetch_document(doc_id)
    chunks = chunk_document(doc.text)
    
    # Embed with new model
    new_embeddings = embed_batch(chunks, model="text-embedding-3-large")
    
    # Upsert into the fork (overwrites old vectors for these doc_ids)
    eval_ns.upsert([
        {
            "id": f"{doc_id}:{i}",
            "vector": emb,
            "text": chunk,
            "doc_id": doc_id,
            "embedding_model": "text-embedding-3-large",
        }
        for i, (chunk, emb) in enumerate(zip(chunks, new_embeddings))
    ])

# Evaluate on the sample
sample_quality = measure_retrieval_quality("my-docs-new-model-eval", eval_set)
print(f"New model recall@10 on 2% sample: {sample_quality['recall_at_10']:.3f}")
print(f"Improvement vs baseline: {sample_quality['recall_at_10'] - baseline['recall_at_10']:+.3f}")

If the sample shows no improvement (or regression), stop here. Discard the fork. Your production namespace is completely unaffected.

Phase 4: Full corpus re-embedding

If the sample validates the new model, proceed to full re-embedding. This is the long-running step:

import asyncio
from tqdm import tqdm

async def re_embed_document(doc_id: str, ns, semaphore: asyncio.Semaphore):
    async with semaphore:
        doc = await fetch_document_async(doc_id)
        chunks = chunk_document(doc.text)
        
        # Embed with new model
        embeddings = await embed_batch_async(chunks, model="text-embedding-3-large")
        
        # Upsert into fork
        await ns.upsert_async([
            {
                "id": f"{doc_id}:{i}",
                "vector": emb,
                "text": chunk,
                "doc_id": doc_id,
                "embedding_model": "text-embedding-3-large",
            }
            for i, (chunk, emb) in enumerate(zip(chunks, embeddings))
        ])

async def full_re_embed(namespace_name: str, all_doc_ids: list[str]):
    ns = client.namespace(namespace_name)
    
    # Limit concurrency to avoid rate limits
    semaphore = asyncio.Semaphore(50)
    
    tasks = [re_embed_document(doc_id, ns, semaphore) for doc_id in all_doc_ids]
    
    # Process with progress bar
    for future in tqdm(asyncio.as_completed(tasks), total=len(tasks)):
        await future

asyncio.run(full_re_embed("my-docs-new-model-eval", all_docs))

Handling production writes during migration

While re-embedding runs, your production namespace continues receiving writes. The fork doesn’t automatically receive these writes.

Two strategies:

Strategy A: Dual-write during migration. Write new documents to both production and the fork simultaneously. Requires code change in your ingestion pipeline but keeps the fork up to date.

Strategy B: Replay writes after migration. Log all writes to production during the migration. After re-embedding completes, replay the log into the fork. Then swap.

For most teams, Strategy B is simpler. The window of drift is the re-embedding duration (typically 2-8 hours for 10M documents at reasonable rate limits).

Phase 5: Validate the full migration

# Full validation after complete re-embedding
full_quality = measure_retrieval_quality("my-docs-new-model-eval", eval_set)
print(f"New model recall@10 (full corpus): {full_quality['recall_at_10']:.3f}")
print(f"Improvement: {full_quality['recall_at_10'] - baseline['recall_at_10']:+.3f}")

# Additional validation: check a random sample of queries
# that users actually ran in the past 7 days
recent_queries = fetch_recent_queries(days=7, n=200)
production_results = run_queries("my-docs", recent_queries)
fork_results = run_queries("my-docs-new-model-eval", recent_queries)

# Manually spot-check results where the ranking changed significantly
diff = find_significant_ranking_changes(production_results, fork_results)
for item in diff[:20]:
    print(f"Query: {item.query}")
    print(f"  Prod top-1: {item.prod_top1}")
    print(f"  New top-1: {item.new_top1}")
    print()

This manual spot-check is important. Aggregate recall numbers can improve while specific high-traffic queries regress. Review the diffs carefully.

Phase 6: Swap to production

If validation passes, swap the production namespace pointer to the fork:

# Create a rollback snapshot before swapping
client.namespace("my-docs").snapshot("pre-model-migration-2026-06-03")

# Atomic swap: production now serves the new model's results
client.namespace("my-docs").swap_with("my-docs-new-model-eval")

print("Migration complete. Production now using text-embedding-3-large.")

The swap is atomic from the query layer’s perspective. In-flight queries against the old namespace complete normally. New queries go to the new index.

Phase 7: Monitor and rollback if needed

Watch your production metrics for 1-2 hours after the swap:

# Watch key metrics
for _ in range(12):  # Check every 10 minutes for 2 hours
    metrics = fetch_production_metrics()
    
    if metrics["p99_latency_ms"] > 50:
        print("ALERT: Latency regression detected")
    
    if metrics["retrieval_quality_score"] < baseline["recall_at_10"] * 0.95:
        print("ALERT: Quality regression detected")
        # Roll back immediately
        client.namespace("my-docs").restore_from("pre-model-migration-2026-06-03")
        print("Rolled back to pre-migration state.")
        break
    
    time.sleep(600)

Rollback is a manifest pointer swap — same as the initial migration, same zero-downtime property. Your corpus goes back to the old model immediately.

Cost accounting

For a 10M document corpus (50M chunks at 1536 dims):

Naive migration cost:

• Old vectors: 50M × 1536 × 4B = 307GB
• New vectors: 307GB (full duplicate during migration)
• Duration of 2x storage: 4-8 hours
• Cost: ~307GB × 2 × $0.023 × (6/730) = $0.06

At S3 prices, even the naive approach is cheap. But copy-on-write makes it essentially free — you’re only paying for the delta (new chunks that differ between old and new model) during the migration, not a full duplicate.

Re-embedding API cost:

• text-embedding-3-large: $0.00013/1K tokens
• At 512 tokens/chunk, 50M chunks: 50M × 512/1000 × $0.00013 = $3,328

The API cost dominates. This is why you evaluate on a sample first — if the new model doesn’t show improvement on 1M chunks, you haven’t wasted $3K re-embedding everything.

Summary

The migration playbook:

1. Baseline: measure current recall before touching anything
2. Fork: create an isolated evaluation environment in milliseconds
3. Sample: validate the new model on 2% of your corpus
4. Full: if the sample validates, re-embed the entire corpus into the fork
5. Validate: compare aggregate recall and spot-check high-traffic queries
6. Swap: atomic promotion with a pre-swap snapshot for rollback
7. Monitor: watch metrics for 2 hours, roll back if needed

The copy-on-write architecture makes steps 2 and 6 fast and storage-efficient. The sample phase makes step 3 cheap enough to try without commitment. The rollback snapshot makes step 7 low-risk.

Embedding model migrations don’t have to be scary. They just need the right primitives.