OpenAI Reworks Habitat Storage for Billion-User Scale
OpenAI's October 23 Habitat Part II describes Azure Cosmos DB design changes for a service it says reaches over one billion users in 40 regions and stores over 500 petabytes. It moved from shared to dedicated containers and back to shared tenancy by default after changing hot-key throttling and serverless scaling.
Habitat's storage design at ChatGPT scale
OpenAI's October 23 engineering post describes Part II of its Habitat online storage story. The company says the platform serves more than one billion users across 40 geographic regions and manages more than 500 petabytes of data. Those figures describe its own system and are not an independent benchmark. Part I covered the transition from Python client library to storage service; Part II focuses on Azure Cosmos DB as the underlying storage component and the operational choices needed at large scale.
The central lesson is a reversal, not a slogan for universal multi-tenancy. Habitat first used a shared Cosmos DB container, then provisioned dedicated containers for workloads that needed isolation, and finally returned to shared tenancy as the default after joint engineering with Azure. Separate containers reduced interference but made each team responsible for capacity planning. A new service could jump from test traffic to one million queries per second, requiring advance provisioning that teams sometimes forgot. The article describes incidents and launch-day scaling pressure caused by that organizational burden.
Why the shared design changed
OpenAI says the revised design throttles the hottest partition keys first so one noisy workload is less likely to consume the throughput of a whole physical partition. An improved Azure Cosmos DB serverless mode supported up to 10,000 request units per second per physical partition and added splitting based on request-unit usage as well as storage. That number is a service design limit reported for OpenAI's arrangement, not an application latency promise. The approach reduces manual pre-scaling but does not eliminate the need to test hot keys and tenant boundaries.
The post says OpenAI's largest Cosmos DB account stores tens of petabytes and serves more than 20 million queries per second. It also recounts a late-2024 regional power outage that exposed failover taking several hours for the largest accounts. The relevant engineering point is that managed storage still requires joint testing at exceptional scale. A cloud provider's ordinary failover documentation cannot replace a full rehearsal with the customer's real data volume and traffic shape.
OpenAI reports read-performance improvements including 12-times faster reads in its storage work, but readers should locate each claim in the company's particular workload and measurement context. This article does not extrapolate those ratios to another deployment. The original post contains more details on caching, containers and the provider partnership; a team planning its own architecture should measure its own read distribution, write pressure and failure behavior before copying one component.
A practical reading of the design
For a smaller service, the useful choice is neither “always share” nor “always isolate.” List tenants by throughput, partition-key distribution, storage growth and failure tolerance. Rehearse a launch spike and a regional failover, including the time to restore useful reads. Track who owns capacity changes and who can slow a hot tenant. OpenAI's experience shows that operational ownership can outweigh a clean theoretical isolation model.
The source is the October 23 Habitat Part II post. This October 26 article reports its architecture and limits. It does not claim access to OpenAI telemetry, Azure service contracts or independent reproduction of the performance numbers.
Sources
FAQ
Why did OpenAI return Habitat to shared tenancy?
OpenAI says dedicated Azure Cosmos DB containers reduced noisy-neighbor problems but pushed launch capacity planning onto each product team. It later worked with Azure on hot-key throttling and improved serverless splitting, allowing shared tenancy to become the default again. This is its specific operational path, not a universal database rule.
What scale does OpenAI report for Habitat?
Its October 23 post says Habitat serves more than one billion users across 40 regions and the broader platform stores over 500 petabytes. It says the largest Cosmos DB account holds tens of petabytes and exceeds 20 million queries per second. These are company-reported infrastructure figures, not independent measurements.
What should another team test before copying the design?
A team should measure its own partition-key heat, tenant interference, launch spikes and regional failover with realistic data. OpenAI recounts a late-2024 outage that exposed hours-long failover for its largest accounts. A managed service still needs joint testing at exceptional scale; the same configuration may not fit smaller workloads.