InfrastructureCompute, chips & cloud
OpenAI moves ChatGPT's Habitat storage layer back to shared Cosmos DB containers co-managed with Microsoft
OpenAI has moved Habitat, the storage layer between ChatGPT and Azure Cosmos DB, back to shared multi-tenant containers co-managed with Microsoft and added global caching, changes it credits with 12× faster ChatGPT reads at more than 70 million requests per second.
From the OpenAI News article of October 23, 2026 and cited sources: Habitat started with one shared Cosmos DB container, then gave each tenant its own container, and has now made shared containers the default again. [1] [7]
SOURCES[1] OpenAI: Rapidly scaling online storage to serve over 1… openai.com[7] Blockchain.News: OpenAI details Habitat storage upgrades blockchain.news
AT Pulse research deskPublished · Updated · 5 min read · AI-assisted analysisAI-assisted, not human-edited. Methodology Corrections Who owns this site
- What happened
- Habitat started with one shared Cosmos DB container, then gave each tenant its own container, and has now made shared containers the default again. [1] [7]
- Original source
- OpenAI News article, company source
- Brief by
- AT Pulse research desk AI-assisted; publisher and contact on the desk page
- When
- October 23, 2026 article published
- Covered
- Oct 24 edition
- Evidence
- 8 sources across 8 sites
- 1OpenAI: Rapidly scaling online storage to serve over 1…openai.com
- 2Azure Cosmos DB Blog: Latest Python SDK, powering AI with…devblogs.microsoft.com
- 3GIGAZINE: OpenAI explains its proprietary storage system…gigazine.net
- 4LLM Rumors: OpenAI Habitat: Storage Scaling, Rust and Tail…llmrumors.com
- 5MemoryLake: OpenAI's Habitat Storage Layermemorylake.ai
- 6HackerNoon: Your Load Test Won't Find These Four Gapshackernoon.com
QuietHow it’s measured
Hacker News · 2 likes
Reach blends public engagement (views, likes, comments, shares, stars) from the platforms where the story or its original source lives: YouTube, Hacker News, GitHub, Substack and similar. A video and the article written from its transcript share one score, taken from the video. Platforms with no data are left out rather than counted as zero, and the age of a story never lowers its score.
Habitat started with one shared Cosmos DB container, then gave each tenant its own container, and has now made shared containers the default again. The new containers are a 'co-managed' type built with Azure's team, combined with global caching, and are meant to keep the protection of isolation while letting tenants share capacity. [1] [7]
Separate containers per tenant prevented many site-wide outages, but they made every product team forecast its own traffic and scale up for launches. In one case a workload went from zero to about 1 million queries per second, roughly 1,000 times its container's baseline. [8]
Habitat reportedly handles more than 70 million requests per second and over 500 PB of data across about 40 regions for more than 1 billion weekly users. Before this storage work, Part I described a rewrite of the Python service in Rust, reported as 6× more CPU-efficient and 15× more memory-efficient. [1] [6] [4] [5]
per-tenant isolation prevented outages but forced teams to forecast capacity, and one workload spiked about 1,000× its baseline.
Read the full assessment
Implication: pooled capacity with tenant controls may cost less to run, though the design depends on Azure-specific container types, raising lock-in questions.
Executive brief
OpenAI says its 12× faster ChatGPT reads came from returning to shared database containers after it had moved away from them. According to OpenAI's Part II post, Habitat, the storage layer between OpenAI products and Azure Cosmos DB, now uses "co-managed" multi-tenant containers built with Microsoft's Azure team, plus global caching. Habitat reportedly handles more than 70 million requests per second and over 500 PB of data for more than 1 billion weekly users. The 12× figure comes only from OpenAI. The reviewed coverage does not include an independent measurement of it.
What changed and event timeline
Microsoft confirms ChatGPT runs on Cosmos DB
Microsoft's Python SDK 4.14.0 announcement says OpenAI relies heavily on Cosmos DB for ChatGPT chat data and that some SDK features were co-developed with OpenAI.
Rust rewrite completed
Habitat's Python service was rewritten in Rust by two engineers working with Codex and GPT-5.5. By September the Rust version handled 95% of requests.
Part I published
OpenAI described the path from client library to service to Rust rewrite and reported 6× CPU and 15× memory efficiency gains. It promised a second post on the storage tier.
More detail
Practitioner analysis of Part I's failure modes
A HackerNoon piece pulled four coordination failures from Part I: event-loop contention, synchronized polling, LIFO connection pools and connection multiplication.
Part II published
OpenAI described multi-tenancy, global caching and co-managed Cosmos DB containers, and claimed 12× faster ChatGPT reads.
More detail
Capabilities and access
- Habitat is an internal platform with no public access.
- It sits in front of Azure Cosmos DB, caches and blob storage, across almost 40 regions.
- Its job now also covers provisioning, developer experience, privacy enforcement and change-data-capture (CDC) feeds into isolated Rockset instances for complex queries (Blockchain.News).
Read the full section
- Habitat is an internal platform with no public access. Its job is to answer requests like "load this user's settings" or "fetch this conversation" (search summary of OpenAI post).
- It sits in front of Azure Cosmos DB, caches and blob storage, across almost 40 regions.
- Its job now also covers provisioning, developer experience, privacy enforcement and change-data-capture (CDC) feeds into isolated Rockset instances for complex queries (Blockchain.News).
- The only public piece is Cosmos DB Python SDK 4.14.0 (Microsoft).
Technical analysis
- How tenancy changed: Habitat started with one shared Cosmos DB container, moved to a separate container per tenant, then went back to shared containers by default (search summary).
- Why separate containers fell short: They prevented many site-wide outages. However, each team had to forecast its own traffic and scale up during launches.
- What replaced them: "Co-managed" containers, built with Azure, try to keep the isolation benefits while sharing capacity.
Read the full section
- How tenancy changed: Habitat started with one shared Cosmos DB container, moved to a separate container per tenant, then went back to shared containers by default (search summary).
- Why separate containers fell short: They prevented many site-wide outages. However, each team had to forecast its own traffic and scale up during launches. One workload jumped from zero to 1 million queries per second, about 1,000× its container's baseline (Superpower Daily).
- What replaced them: "Co-managed" containers, built with Azure, try to keep the isolation benefits while sharing capacity.
- Reproducibility: The reviewed sources do not document the 12× read benchmark: no baseline, latency percentile or workload is given.
Claims and evidence
Conflicting figure: GIGAZINE's headline says 40 PB, but its own body says more than 500 PB (GIGAZINE).
Read the full section
| Claim | Source |
| 12× faster ChatGPT reads | OpenAI only (post). Blockchain.News notes there are no independently verified measurements (link). |
| More than 70M requests/sec, 500+ PB, about 40 regions | OpenAI, repeated in coverage (HackerNoon) |
| Rust is 6× more CPU-efficient and 15× more memory-efficient | OpenAI. LLM Rumors says the ratios lack a reproducible methodology (link). |
| ChatGPT chat data is stored in Cosmos DB | Microsoft (blog) |
Conflicting figure: GIGAZINE's headline says 40 PB, but its own body says more than 500 PB (GIGAZINE).
Context and prior work
- Part I covered the compute side: the Python service peaked above 20 million requests per second, and traffic grew about 10× a year for three years (HackerNoon).
- Part II covers the storage tier.
- The OpenAI–Azure engineering ties go back at least to the October 2025 SDK work (Microsoft).
Read the full section
- Part I covered the compute side: the Python service peaked above 20 million requests per second, and traffic grew about 10× a year for three years (HackerNoon).
- Part II covers the storage tier.
- The OpenAI–Azure engineering ties go back at least to the October 2025 SDK work (Microsoft).
- Co-managed containers continue that partnership. They are a vendor-specific design, not a general storage technique.
Limitations, safety and contested findings
- Noisy neighbours: Shared containers bring back the risk that one tenant's traffic slows others. Separate containers had limited the damage from such failures (Superpower Daily).
- Limits of scaling out: Hot partitions in Cosmos DB cannot be fixed just by adding capacity, and requests that fan out to many parallel reads make slow tail latency more likely (LLM Rumors).
- Efficiency claims: The reviewed sources show no independent reproduction of any of OpenAI's efficiency or speed figures.
Read the full section
- Noisy neighbours: Shared containers bring back the risk that one tenant's traffic slows others. Separate containers had limited the damage from such failures (Superpower Daily).
- Limits of scaling out: Hot partitions in Cosmos DB cannot be fixed just by adding capacity, and requests that fan out to many parallel reads make slow tail latency more likely (LLM Rumors).
- Efficiency claims: The reviewed sources show no independent reproduction of any of OpenAI's efficiency or speed figures.
Business and practitioner implications
- Isolation has a planning cost. A shared pool with good tenant controls can be cheaper to run.
- Hyperscale may need vendor co-engineering. OpenAI's result depended on container types designed with Microsoft. That raises lock-in questions for anyone copying the pattern.
- Test coordination failures, not just load. Synchronized polling and connection-pool feedback loops are worth checking before chasing database speedups (HackerNoon).
Read the full section
- Isolation has a planning cost. Giving each team its own container pushes capacity forecasting onto product teams, which is costly when launches spike unpredictably. A shared pool with good tenant controls can be cheaper to run.
- Hyperscale may need vendor co-engineering. OpenAI's result depended on container types designed with Microsoft. That raises lock-in questions for anyone copying the pattern.
- Test coordination failures, not just load. Synchronized polling and connection-pool feedback loops are worth checking before chasing database speedups (HackerNoon).
- Treat the 12× as directional. Without published baselines it should not be used as a planning benchmark.