Migrating from Pinecone to ManyVector

Migrating a vector database in production is sensitive. Your embedding data is irreplaceable, and any disruption to query serving causes downstream failures. This guide walks through a safe, zero-downtime migration from Pinecone to ManyVector.

Why teams migrate

Common reasons teams switch to ManyVector:

• Cost: Pinecone charges for pod size (always-on RAM). ManyVector charges per query. At typical production workloads, teams see 5-10x cost reduction.
• Data sovereignty: Pinecone stores your vectors on Pinecone’s infrastructure. ManyVector stores them in your own S3/GCS/R2 bucket.
• Namespace branching: Pinecone has no equivalent to ManyVector’s copy-on-write namespace forks.
• No pod capacity planning: ManyVector scales transparently with your object store — no pod type selection, no capacity planning.

Migration strategy

We recommend a four-phase approach:

1. Export — export all vectors from Pinecone
2. Import — upsert into ManyVector (while Pinecone stays live)
3. Validate — compare recall between both systems on a ground-truth eval set
4. Cut over — switch application traffic to ManyVector; keep Pinecone read-only for 7 days

Phase 1: Export from Pinecone

Pinecone does not offer a bulk export endpoint — you fetch vectors by ID or paginate via query:

import pinecone
import json

pc = pinecone.Pinecone(api_key="pc-...")
index = pc.Index("my-index")

# Option 1: Export by ID list (if you have stored IDs)
def export_by_ids(ids: list[str], output_path: str):
    all_vectors = []
    batch_size = 100

    for i in range(0, len(ids), batch_size):
        batch = ids[i : i + batch_size]
        response = index.fetch(ids=batch)
        for id_, data in response.vectors.items():
            all_vectors.append({
                "id": id_,
                "vector": data.values,
                "metadata": dict(data.metadata or {}),
            })

    with open(output_path, "w") as f:
        for v in all_vectors:
            f.write(json.dumps(v) + "\n")

    print(f"Exported {len(all_vectors)} vectors to {output_path}")

If you don’t have a stored ID list, query with a zero vector and paginate (Pinecone’s starter index doesn’t support list — use a serverless index):

# For Pinecone serverless: list all vector IDs
def list_all_ids() -> list[str]:
    ids = []
    for id_batch in index.list(limit=100):
        ids.extend(id_batch)
    return ids

Phase 2: Import into ManyVector

Create a ManyVector namespace matching your Pinecone index configuration:

from manyvector import ManyVector

mv = ManyVector(
    api_key="mv-...",
    backend={"provider": "s3", "bucket": "my-vectors", "region": "us-east-1", ...}
)

# Create namespace matching Pinecone index dimensions and metric
ns = mv.create_namespace(
    name="my-index",
    dimensions=1536,      # must match Pinecone index dimension
    metric="cosine",      # must match Pinecone metric
)

Import the exported vectors:

import json

def import_vectors(export_path: str, batch_size: int = 200):
    batch = []
    total = 0

    with open(export_path) as f:
        for line in f:
            record = json.loads(line)
            batch.append({
                "id": record["id"],
                "vector": record["vector"],
                **record["metadata"],  # all metadata fields become ManyVector attributes
            })

            if len(batch) >= batch_size:
                ns.upsert(batch)
                total += len(batch)
                batch = []
                print(f"Imported {total} vectors...")

    if batch:
        ns.upsert(batch)
        total += len(batch)

    print(f"Import complete: {total} total vectors")

import_vectors("pinecone_export.jsonl")

Phase 3: Validate recall parity

Before cutting over traffic, compare recall between Pinecone and ManyVector on your eval set:

def pinecone_query(query_vector: list[float], top_k: int = 10) -> set[str]:
    result = index.query(vector=query_vector, top_k=top_k, include_metadata=False)
    return {m.id for m in result.matches}

def manyvector_query(query_vector: list[float], top_k: int = 10) -> set[str]:
    result = ns.query(vector=query_vector, top_k=top_k)
    return {r.id for r in result}

# Compare on ground-truth eval set
agreement_scores = []
for item in eval_set:
    pc_ids = pinecone_query(item["vector"])
    mv_ids = manyvector_query(item["vector"])
    # Measure overlap between both result sets
    agreement = len(pc_ids & mv_ids) / len(pc_ids)
    agreement_scores.append(agreement)

avg_agreement = sum(agreement_scores) / len(agreement_scores)
print(f"Avg result overlap between Pinecone and ManyVector: {avg_agreement:.2%}")

Expect 85-95% overlap — HNSW is approximate, so some divergence is normal. If overlap is below 80%, check that dimensions and distance metric match exactly.

Phase 4: Cut over

Use a feature flag to gradually shift traffic:

import random

def vector_query(query_vector: list[float], top_k: int = 10, rollout_pct: int = 0):
    """
    rollout_pct: percentage of traffic to send to ManyVector (0-100)
    """
    use_manyvector = random.randint(0, 99) < rollout_pct

    if use_manyvector:
        return manyvector_query(query_vector, top_k)
    else:
        return pinecone_query(query_vector, top_k)

Rollout schedule:

Day	ManyVector traffic
Day 1	5%
Day 2	20%
Day 3	50%
Day 5	100%

Monitor error rates and p99 latency at each step. If metrics degrade, roll back to 0% instantly.

Pinecone metadata → ManyVector attributes

Pinecone stores metadata as a flat JSON object. ManyVector stores attributes alongside vectors and uses them for both BM25 full-text search (on text fields) and metadata filtering:

# Pinecone metadata
{
  "text": "ManyVector stores HNSW indexes in object storage.",
  "source": "docs",
  "published_date": "2026-01-15",
  "score": 0.95
}

# ManyVector equivalent — same structure, works with filters and BM25
ns.upsert([{
    "id": "doc-123",
    "vector": [...],
    "text": "ManyVector stores HNSW indexes in object storage.",  # BM25-indexed
    "source": "docs",           # filterable: {"source": {"$eq": "docs"}}
    "published_date": "2026-01-15",
    "score": 0.95,
}])

Post-migration cleanup

After 7 days of stable operation on ManyVector:

1. Stop all writes to Pinecone
2. Export a final snapshot from Pinecone as a backup
3. Delete the Pinecone index (saves on pod costs immediately)
4. Update all documentation and runbooks

Cost comparison

Cost item	Pinecone (s1 pod, 1M vectors)	ManyVector
Storage	~$70/mo per pod	~$0.023/GB/mo in S3
Queries	Included in pod	$0.00004/query above free tier
Writes	Included in pod	Free
Total (1M vectors, 1M queries/mo)	$70-140/mo	~$5-10/mo

Actual savings depend on your query volume and vector dimensions. Most teams with >5M vectors and >1M queries/month see 5-10x cost reduction.