The hidden cost of in-memory vector databases

The pricing page for every major vector database is carefully designed. It shows you a low starting price. It might show a free tier. It probably shows a comparison table that positions itself favorably.

What it doesn’t show you is the full cost of the architecture you’re buying into.

This post isn’t a hit piece on any particular vendor. It’s an accounting exercise: what does a RAM-first vector database actually cost to operate, and where do the hidden expenses come from?

The RAM tax

Every vector that lives in a managed vector database lives in RAM. This is a fundamental property of HNSW and most other ANN index structures — they require random access into the index, which requires the index to be in memory.

The economics of RAM in 2026:

• AWS r7g.16xlarge: $4.83/hr, 512GB RAM
• Amortized over a month: ~$3,478
• Per GB of RAM: $6.79/GB/month

Compare to S3 Standard: $0.023/GB/month.

That’s a 295x cost difference for the storage medium. Managed vector databases can’t escape this cost — they can only amortize it across customers and take a margin. Their pricing reflects RAM economics, not storage economics.

The replication multiplier

RAM is volatile. Every cloud provider runs at least 3x replication for production data. Your vector database provider does too — they have to, for durability and availability.

3x replication means every GB of your data costs 3x in RAM. If you store 100GB of vectors, the vendor is paying for 300GB of RAM to serve your data. This multiplier is baked into pricing but rarely shown explicitly.

Object storage handles replication differently. S3, GCS, and R2 replicate your data across multiple availability zones and their pricing already includes this replication. You pay $0.023/GB for 11 nines of durability. There’s no “replication surcharge.”

The idle capacity tax

Managed vector databases charge you continuously — whether or not you’re running queries. Your vectors have to be resident in RAM at all times because loading from disk takes too long at query time.

Consider a SaaS product with 10,000 users and a per-user knowledge base of 10,000 vectors each. That’s 100M vectors total. But on any given hour, only 100-500 users are actively querying (generous estimate for a business-hours product). The other 9,500+ users’ vectors are sitting idle in RAM, costing money.

With an object-storage-native approach, cold namespaces don’t occupy RAM. You pay S3 prices for idle data. Only queried namespaces warm up — and once warm, they stay warm in a cache for as long as they’re being queried.

The idle capacity tax is where the pricing model really diverges for multi-tenant products.

Scaling cliffs

RAM-first systems scale in discrete jumps. You’re running a 3-node cluster with 128GB RAM per node (384GB total). Your dataset grows to 350GB. You have two options:

1. Vertical scale: move to larger nodes. You’re now paying for 512GB nodes — 46% more capacity, immediately, whether you need all of it or not.

2. Horizontal scale: add a fourth node. More expensive, requires rebalancing, may require downtime or at minimum a brief performance degradation.

Neither option is graceful. You’re paying for headroom you don’t yet need because the alternative is running out of capacity.

Object storage scales to exabytes without a configuration change. You upsert another million vectors — they go to S3, your bucket grows, you pay $0.023/GB. No cluster management, no rebalancing, no capacity planning beyond “how much do we want to store.”

Egress: the bill that surprises everyone

Cloud egress fees are the industry’s dirtiest secret. AWS charges $0.09/GB for data transferred out of S3 to the internet. GCS: $0.08-0.12/GB. Azure: $0.05-0.087/GB.

Managed vector databases add another layer: egress from the vector database to your application. If your vector database runs in a different cloud or region than your application (common in multi-cloud setups), you pay egress twice.

For RAG pipelines that retrieve 10 chunks per query at ~2KB each:

• Per query: 20KB of data returned
• At 10M queries/month: 200GB of data returned
• At $0.09/GB AWS egress: $18/month just in egress

This isn’t catastrophic at 10M queries, but it compounds. At 100M queries/month, you’re paying $180/month just to move data — on top of the storage and query fees.

With ManyVector, query results travel from your S3 bucket (in your account) through the ManyVector query layer and back to your application. If your S3 bucket is in the same region as your application, egress is free (AWS doesn’t charge for within-region data transfer). If you use Cloudflare R2, egress is free globally.

The lock-in cost

This one is harder to quantify but real: migrating away from a managed vector database is expensive.

Your vectors are stored in proprietary format on the vendor’s infrastructure. To migrate, you typically need to:

1. Export all vectors (if the vendor supports export — not all do)
2. Re-import into the new system
3. Validate retrieval quality
4. Cut over traffic

For 100M vectors, this is a significant operational project. The friction keeps teams on underperforming or overpriced systems long past the point where switching makes economic sense.

With ManyVector, your data is in your S3 bucket, in an open format. Migrating away means migrating a bucket — or just pointing a different system at the same bucket. There’s no vendor export process.

A realistic cost comparison at scale

Let’s price a concrete workload: 500M vectors, 1536 dimensions, 50M queries/month.

Pinecone (estimated):

• Storage (serverless): ~$300-400/month
• Queries at 50M/month: ~$800-1,200/month
• Total: ~$1,100-1,600/month

Weaviate Cloud:

• 3TB HNSW index (with overhead): needs premium tier
• Estimated: ~$1,500-2,500/month

Qdrant Cloud:

• Cluster needed for 3TB+ in-memory: 6+ nodes
• Estimated: ~$1,800-3,000/month

ManyVector on S3:

• Vector storage (500M × 1536 dims × 4 bytes = 3TB): $69/month (S3 Standard)
• S3 GET costs: ~$25/month
• ManyVector query fees: usage-based
• Total storage + GET: ~$94/month + query fees

Even adding a modest query fee, the total is 5-10x lower than the incumbents. The difference grows with scale because the S3 storage cost curve is flat while RAM-based pricing grows faster than linearly (you pay for headroom).

What you give up

Honesty requires acknowledging the tradeoff: in-memory vector databases have consistently low latency for any query against any namespace, hot or cold. There’s no cold-start penalty because nothing is ever cold.

If your workload is a single massive namespace queried at constant high QPS, an in-memory database is appropriate. The hot/warm/cold distinction doesn’t apply when everything is always hot.

For multi-tenant applications, batch RAG pipelines, or any workload with a natural hot/cold split — object-storage-native is the better economic choice by a wide margin.

The architectural question

The fundamental question isn’t “which vector database is cheaper.” It’s “what’s the right architecture for a world where storage is 300x cheaper than RAM?”

The answer the incumbent vector databases give is: keep vectors in RAM because that’s what the algorithm needs. The answer ManyVector gives is: move the caching problem to the query layer, not the storage layer. Keep hot namespaces in RAM. Let cold namespaces live in S3. Pay RAM prices only for what you’re actively querying.

This is the same architectural insight that made columnar databases on S3 (Athena, BigQuery, Redshift Spectrum) successful against traditional on-disk data warehouses. The storage medium got cheap enough that the economics of always-on compute couldn’t compete.

Vector databases are at the same inflection point.