Uber Writes A Data Store To Save 6 Million

Uber built a purpose-built long-term data store, Ledger store, to take over payment transaction storage from Dynamo DB and blob storage—cutting annual maintenance costs by an estimated $6 million while migrating roughly 1 trillion records. The shift matters because it targets one of the hardest problems in fintech-scale systems: keeping financial transaction data both complete and correct over time, across many access patterns, without letting storage costs and operational complexity spiral.

Before Ledger store, Dynamo DB handled only “hot” payment data up to about 12 weeks old, while older data lived in an in-house blob service. Rising storage costs pushed Uber to reduce Dynamo DB usage, and the new architecture extends that cost-control strategy into a dedicated system designed for financial integrity. Ledger store is described as immutable storage that provides verifiable completeness and correctness guarantees—treating ledgers as the source of truth for financial events and data movement.

The core engineering challenge is indexing at massive scale. Uber needs to look up ledgers via many access patterns, which drives “trillions of indexes” over “hundreds of billions of ledgers.” Ledger store therefore relies on strongly consistent indexing, supported by a two-phase commit approach: it persists an intent in the index first, then writes the record, and finally asynchronously commits or rolls back depending on success. The transcript also emphasizes the operational reality of managing those indexes over time, including index life cycle management—automated reindexing when index definitions change. That process creates a new index, backfills data, validates, swaps to the new index, and deletes the old one.

Cost and performance improvements are paired with migration safety. Uber used “shadow validation” by double-writing payment data to both Dynamo DB and Ledger store, then comparing read results to confirm a near one-to-one surface before fully switching. For the migration itself, the team used shallow and offline validation, plus historical data validation and an incremental backfill job running in Apache Spark. The backfill load reportedly reached about 10x normal production load and took around three months.

To reduce risk during rollout, Uber took a conservative deployment strategy with a fallback path: if data wasn’t found in Ledger store, systems could fetch it from Dynamo DB. The migration reportedly completed without downtime or outages. Overall savings were estimated at more than $6 million per year, attributed to reduced reliance on Dynamo DB and blob storage and the simplified long-term storage architecture—enough, in the transcript’s framing, to fund roughly a handful of additional engineers.

The discussion around why build instead of buy centers on the mismatch between generic distributed databases and the specific requirements of immutable financial ledgers, verifiable integrity, and strongly consistent indexing at Uber’s scale. Even with skepticism about reinventing storage, the transcript repeatedly returns to the same point: when data volume, correctness requirements, and indexing demands reach extreme levels, bespoke engineering can become the only way to meet both integrity and cost targets.