Object storage 101: S3, GCS, R2, and MinIO compared

ManyVector stores all vector data — HNSW indexes, BM25 inverted indexes, and raw vector objects — in your own object store. Which object store should you use? This guide compares the major options: AWS S3, Google Cloud Storage, Cloudflare R2, and MinIO.

What is object storage?

Object storage is a flat storage architecture where data is stored as objects in buckets. Each object has a key (like a file path), data (the content), and metadata. There’s no directory hierarchy, no locking, no transactions — just PUT and GET operations at massive scale.

Object storage’s key properties make it well-suited for vector indexes:

• Infinite scale: S3, GCS, and R2 scale to exabytes. You never run out of capacity.
• High durability: S3 Standard guarantees 11 nines (99.999999999%) object durability.
• Low cost: $0.015-$0.023/GB/month — orders of magnitude cheaper than NVMe or RAM.
• Pay-per-use: No idle capacity cost. You pay for what you store and what you access.

AWS S3

S3 (Simple Storage Service) is the original cloud object store, launched in 2006. It’s the most widely adopted and the de facto standard — every region, every cloud tool, every language SDK supports it.

Storage classes:

Class	Cost	Durability	Use case
S3 Standard	$0.023/GB/mo	99.999999999%	Hot namespaces
S3 Standard-IA	$0.0125/GB/mo	99.999999999%	Warm namespaces
S3 Express One Zone	$0.016/GB/mo + faster ops	99.99%	Ultra-low latency hot data
S3 Glacier Instant	$0.004/GB/mo	99.999999999%	Cold/archive namespaces

Operations pricing (S3 Standard, us-east-1):

• PUT/COPY/POST: $0.005 per 1,000 requests
• GET: $0.0004 per 1,000 requests

Strengths:

• Largest ecosystem of tooling, integrations, and documentation
• S3 Express One Zone offers lowest latency for hot data (single-digit ms read latency)
• Comprehensive lifecycle policies, versioning, and replication
• 100+ compliance certifications (SOC 2, ISO 27001, HIPAA, FedRAMP, etc.)
• IAM integration with fine-grained access controls

Weaknesses:

• Egress fees: $0.09/GB for data transferred out of AWS (to internet or other regions)
• Most complex pricing model (storage + operations + request + data transfer)

Best for: Production workloads already on AWS, compliance-sensitive deployments, when ecosystem tooling matters.

Google Cloud Storage

GCS is Google’s object store, with strong integration into the GCP ecosystem (BigQuery, Vertex AI, Dataflow). It’s competitive with S3 on price and durability.

Storage classes:

Class	Cost (US)	Use case
Standard	$0.020/GB/mo	Hot namespaces
Nearline	$0.010/GB/mo	Accessed <1/month
Coldline	$0.004/GB/mo	Accessed <1/quarter
Archive	$0.0012/GB/mo	Long-term retention

Strengths:

• Dual-region and multi-region configurations for higher availability
• Strong consistency (all operations are strongly consistent since 2021)
• Excellent integration with Vertex AI for ML workloads
• CMEK with Cloud KMS for customer-managed encryption keys
• Better pricing than S3 at multi-region scale

Weaknesses:

• Egress fees apply outside GCP (similar to S3)
• Smaller ecosystem than S3 (though S3-compatibility layer exists)
• Operations pricing can add up for write-heavy workloads

Best for: GCP-native architectures, ML pipelines on Vertex AI, teams already on GCP, multi-region deployments.

Cloudflare R2

R2 is Cloudflare’s object store with S3-compatible API. Launched in 2022, it has grown quickly and has one unique property: zero egress fees.

Pricing:

Item	Cost
Storage	$0.015/GB/mo
Class A operations (PUT, POST)	$4.50 per million
Class B operations (GET)	$0.36 per million
Egress	$0.00

Strengths:

• Zero egress fees — globally distributed reads at no data transfer cost
• Cheapest storage per GB among the major cloud providers
• S3-compatible API — works with existing S3 tooling and SDKs
• Jurisdiction controls (EU, APAC) for data residency
• Globally distributed — reads are served from nodes close to the requester

Weaknesses:

• No storage tiering (one class only)
• Smaller ecosystem and fewer compliance certifications than AWS/GCP
• Operations pricing can be higher than S3 for write-heavy workloads

Best for: Globally distributed deployments where egress costs matter, teams with write-once/read-many workloads, cost-sensitive deployments.

MinIO

MinIO is an open-source, high-performance, S3-compatible object store. Unlike the cloud options, MinIO runs on your own hardware or VMs — on-premises, in a private cloud, or air-gapped.

Pricing: No licensing cost (open-source). You pay for your own hardware, bandwidth, and operations team.

Performance: MinIO on NVMe delivers extraordinary performance — benchmarks show 325 GiB/s GET and 165 GiB/s PUT throughput in distributed mode. For hot vector workloads where every millisecond counts, on-premises MinIO on NVMe can outperform cloud object storage.

Strengths:

• Full control and data sovereignty — data never leaves your infrastructure
• S3-compatible — ManyVector connects using the same client as AWS S3
• No egress fees — everything stays on your network
• Air-gapped operation — works with no internet connectivity
• AGPL-licensed, commercially supported by MinIO Inc.

Weaknesses:

• Operational overhead — you’re responsible for hardware, deployment, scaling, and reliability
• Lower durability ceiling than cloud providers (depends on your RAID/erasure coding configuration)
• No managed compliance certifications (you certify your own deployment)

Best for: Regulated industries requiring on-premises data, air-gapped environments, teams with existing hardware infrastructure, cost optimization at large scale.

Decision guide

Start with S3 if:

• You’re already on AWS or don’t have a strong preference
• Compliance certifications matter (SOC 2, HIPAA, FedRAMP)
• You need the broadest ecosystem support

Choose GCS if:

• You’re already on GCP
• You’re running ML pipelines on Vertex AI
• You want multi-region redundancy with better pricing than S3 multi-region

Choose R2 if:

• Egress fees are significant in your architecture
• You have globally distributed query nodes
• Cost optimization is the primary concern

Choose MinIO if:

• You need on-premises or air-gapped operation
• Data sovereignty requires that data never leaves your infrastructure
• You have the operational capacity to manage your own storage

Connecting to ManyVector

All four options work with ManyVector. The configuration follows the same pattern:

from manyvector import ManyVector

# AWS S3
client = ManyVector(api_key="mv-...", backend={
    "provider": "s3",
    "bucket": "my-vectors",
    "region": "us-east-1",
})

# Google Cloud Storage
client = ManyVector(api_key="mv-...", backend={
    "provider": "gcs",
    "bucket": "my-vectors",
    "project_id": "my-gcp-project",
})

# Cloudflare R2
client = ManyVector(api_key="mv-...", backend={
    "provider": "r2",
    "bucket": "my-vectors",
    "account_id": "your-cf-account-id",
})

# MinIO
client = ManyVector(api_key="mv-...", backend={
    "provider": "s3_compatible",
    "endpoint": "http://minio.internal:9000",
    "bucket": "vectors",
})

Cost comparison example

For 1 billion vectors at 1536 dimensions (float32) ≈ 5.7TB raw data + ~1.5TB HNSW overhead = ~7.2TB total:

Provider	Monthly storage	Notes
S3 Standard	~$165/mo	Plus egress if applicable
GCS Standard (US)	~$144/mo	Plus egress if applicable
Cloudflare R2	~$108/mo	Zero egress
MinIO	Hardware cost	Amortized over hardware lifetime
ManyVector markup	$0	You pay your provider directly

None of these include ManyVector’s query pricing — which charges per query, not per GB stored.