Why the Smartest AI Teams Are Panic-Buying Compute: The 36-Month AI Infrastructure Crisis Is Here

AI-driven demand for inference compute is colliding with a physically constrained hardware supply chain, creating a structural “36-month” infrastructure crisis that will push inference costs sharply higher and force enterprises to rethink how they plan, buy, and run AI. The central warning: this isn’t a temporary tech shortage. It’s an economic transformation—demand keeps accelerating while memory, advanced chips, and GPU allocation can’t scale fast enough—so competitive dynamics across industries will shift as budgets tighten and capacity becomes scarce.

The pressure starts with consumption. Heavy enterprise users can burn through roughly a billion tokens per worker per year, while “ceiling” usage for intense workloads can reach 25 billion tokens annually. Token growth isn’t just linear; it accelerates as model capability improves and as AI becomes embedded across everyday software—email, document editors, development environments, and CRM—turning AI from a tool into ambient, continuous consumption. The shift toward agentic systems compounds the effect: agents can run continuously and call other AI in loops, consuming orders of magnitude more tokens than human-in-the-loop workflows. At enterprise scale, the math becomes stark: a 10,000-person company spending about $20 million per year on inference at 1 billion tokens per worker could jump to $200 million at 100 billion tokens per year, and those figures assume stable pricing and available capacity—assumptions the crisis directly breaks.

On the supply side, the bottleneck is memory and advanced semiconductor capacity. DRAM fabrication takes 3–4 years and capacity is already fully allocated; high bandwidth memory (HBM) is effectively sold out and can’t be substituted at scale. DRAM prices are projected to surge—Trend Force forecasts memory costs adding 40–60% to inference infrastructure in the first half of 2026, with effective inference costs potentially doubling or tripling within 18 months. Even if DRAM supply improves, the advanced chip layer is constrained: TSMC’s leading nodes are fully allocated, with capacity expansion timelines stretching to 2028 and beyond. GPU availability adds another choke point. Nvidia’s H100 and newer Blackwell GPUs dominate AI workloads and are sold out, with lead times exceeding six months and multi-year hyperscaler purchase agreements locking up most production.

A key twist is that cloud providers aren’t neutral capacity brokers. AWS, Azure, Google Cloud, and similar players also sell competing AI products (Gemini, Copilot, AWS AI services). When compute is scarce, allocation becomes zero-sum: GPUs sent to enterprises mean fewer GPUs for internal products. Hyperscalers are therefore incentivized to prioritize their own strategic workloads, tightening enterprise rate limits and making allocation commitments harder.

The result is a market that won’t “clear” smoothly. Supply can’t respond quickly, demand can’t be deferred, and information is asymmetric—hyperscalers understand constraints better than most enterprises. Pricing spikes are likely to be abrupt rather than gradual, echoing past volatility in DRAM during the 2016 shortage and GPU surges during crypto mining.

For enterprise leaders, traditional capex planning and depreciation schedules fail because demand and hardware usefulness change faster than procurement cycles. The transcript’s practical playbook centers on four moves: secure inference capacity now via throughput/availability SLAs (not just price-per-token), build a routing layer that can shift workloads across providers and models to preserve optionality, treat hardware as a consumable with accelerated refresh/lease strategies, and invest aggressively in inference efficiency (smaller models where possible, better prompting, caching, retrieval augmentation, quantization). The overarching message is that winners will be those who act within the shrinking window to lock capacity and reduce cost exposure before the crisis peaks.