Copy-on-write namespaces: Git for your vector database

Every software team uses Git. The core insight of Git isn’t version control in the abstract — it’s copy-on-write branching. Creating a branch doesn’t copy your codebase. It creates a pointer to the current commit. You only pay for new data when you write new data.

This is a profoundly useful primitive. It’s the reason you can have 50 feature branches without running out of disk. It’s why reverting is cheap. It’s why experiments are low-risk — you can try something, see that it’s wrong, and discard it without affecting anything else.

Vector databases have needed this for a long time.

The problem with mutable namespaces

A namespace (or collection, or index — terminology varies) in a vector database is typically a mutable bag of vectors. You upsert into it. You delete from it. It has one state at any given time.

This works fine until you want to do any of the following:

Evaluate a new embedding model. Your RAG pipeline uses text-embedding-3-small. You want to test text-embedding-3-large. To do this properly, you need to re-embed your entire corpus with the new model and run queries against both indexes to compare recall metrics. With a mutable namespace, you have two choices: maintain two full copies of the data (expensive) or modify your production namespace (terrifying).

Run A/B tests on retrieval. You want to compare HNSW with M=16 vs M=32, or cosine similarity vs dot product, or different chunking strategies. Each variant is a different state of the namespace. You can’t easily A/B test against a single mutable index.

Protect against bad writes. A bug in your embedding pipeline writes malformed vectors to your production namespace. You have no clean state to revert to. Your only option is to re-ingest everything from scratch.

Stage changes before promoting. You’re updating 10% of your corpus. You want to verify retrieval quality before the new embeddings go live. With a mutable namespace, there’s no staging concept — you write directly to production.

Copy-on-write namespaces

The solution is to apply Git’s branching model to vector storage.

A namespace in ManyVector is backed by a set of immutable files on object storage. These files are content-addressed — their names are derived from their content hash. When you upsert vectors, ManyVector doesn’t modify existing files. It writes new files and updates a lightweight namespace manifest that records which files compose the current state.

The manifest looks roughly like:

{
  "namespace": "my-docs",
  "version": "sha256:a3b4c5...",
  "segments": [
    { "id": "seg-001", "path": "s3://my-bucket/namespaces/my-docs/seg-001.hnsw", "vectors": 50000 },
    { "id": "seg-002", "path": "s3://my-bucket/namespaces/my-docs/seg-002.hnsw", "vectors": 50000 }
  ],
  "deleted": []
}

Forking a namespace creates a new manifest that points to the same segment files as the parent. No data is copied. The fork is instantaneous regardless of how many vectors are in the namespace.

# Fork a 100M-vector namespace — completes in milliseconds
fork = client.namespace("my-docs").fork("my-docs-experiment")

The forked namespace starts as an alias for the parent. As you upsert new vectors into the fork, ManyVector writes new segment files and updates the fork’s manifest. The parent namespace is untouched. The new files are the only storage overhead — you don’t pay for the shared segments twice.

Practical applications

Embedding model evaluation

The workflow that would previously require duplicating your entire dataset now looks like:

# Step 1: Fork production namespace
eval_ns = client.namespace("my-docs").fork("my-docs-eval-large-model")

# Step 2: Re-embed only a sample for initial evaluation
sample_docs = load_sample_docs(n=10000)
new_embeddings = embed(sample_docs, model="text-embedding-3-large")
eval_ns.upsert(new_embeddings)

# Step 3: Compare recall on your eval set
prod_results = client.namespace("my-docs").query(eval_queries)
eval_results = eval_ns.query(eval_queries)
compare_recall(prod_results, eval_results)

# Step 4: If the new model wins, re-embed the full corpus into the fork
# Step 5: Swap the production namespace pointer to the fork
client.namespace("my-docs").swap_with("my-docs-eval-large-model")

The swap_with operation is a manifest pointer swap — atomic, instant. Zero downtime.

Safe batch updates

# Fork before a risky batch update
staging = client.namespace("my-docs").fork("my-docs-staging")

# Apply updates to staging
for batch in update_batches:
    staging.upsert(batch)

# Run quality checks against staging
assert retrieval_quality(staging) > threshold

# Promote staging to production
client.namespace("my-docs").swap_with("my-docs-staging")

If the quality check fails, discard the fork. Your production namespace is completely unaffected.

Per-user or per-tenant namespaces

Multi-tenant RAG applications often need per-user knowledge bases that start from a shared base. With copy-on-write forking, you can maintain a “base” namespace for shared company knowledge and fork it per user:

def get_user_namespace(user_id: str):
    ns_name = f"user-{user_id}"
    if not client.namespace_exists(ns_name):
        # Fork from shared base — instantaneous
        client.namespace("company-base").fork(ns_name)
    return client.namespace(ns_name)

Each user’s namespace starts with the shared company data (shared storage) and accumulates user-specific vectors independently (incremental storage only). A company with 10,000 users doesn’t pay for 10,000 copies of the base namespace — they pay for the base once, plus the incremental user-specific data.

Rollback

# Create a named snapshot before a risky migration
client.namespace("my-docs").snapshot("pre-migration-2026-07-01")

# Run migration
run_migration()

# Something went wrong — roll back
client.namespace("my-docs").restore_from("pre-migration-2026-07-01")

restore_from swaps the manifest pointer back to the snapshot. Data that was written during the migration remains on object storage until garbage collected, but the namespace immediately reflects the pre-migration state.

Why object storage makes this possible

Copy-on-write branching is a natural fit for object storage because object storage is already immutable at the object level. You can’t modify an S3 object in place — you write a new object with the same key (which is a replace, not a mutation) or a new key.

ManyVector’s architecture treats this as a feature: all segment files are content-addressed and never modified. Forking is just creating a new manifest. Writes go to new objects. The shared segments from the parent are referenced by multiple manifests simultaneously — no copying, just reference counting.

In-memory vector databases can’t do this efficiently. Sharing memory between “branches” of an HNSW index requires careful reference counting of mutable data structures. The complexity is high and the implementation is error-prone. The object storage model makes the shared-nothing isolation free.

Garbage collection

One obvious question: what happens to old segments that no longer appear in any active manifest?

ManyVector runs a background garbage collector that scans manifests and marks orphaned segments for deletion. By default, segments are retained for 24 hours after they become orphaned (to allow restoring from recent history). You can configure this per-namespace:

client.namespace("my-docs").set_retention(hours=72)  # Keep 3 days of history
client.namespace("my-docs").set_retention(hours=0)   # Delete orphans immediately

The GC runs entirely within your bucket — no data is sent anywhere. Deleted segments become standard S3 delete operations.

Summary

Copy-on-write namespace branching treats your vector database the way Git treats your codebase. The primitives map cleanly:

Git	ManyVector
git checkout -b feature	namespace.fork("feature")
git commit	namespace.upsert(vectors)
git revert	namespace.restore_from(snapshot)
Merge branch	namespace.swap_with(fork)
git tag	namespace.snapshot(name)

The storage cost model is the same too: you pay for new data, not for the full namespace per branch. A 100M-vector namespace with 10 active experiment branches costs storage for 100M vectors plus whatever new vectors were written into each branch.

For teams running embedding model evaluations, A/B tests, or batch migrations, this changes the risk profile of every experiment. Try things. They’re cheap to discard.