Leak: ‘GPT-5 exhibits diminishing returns’, Sam Altman: ‘lol’

A leaked account of OpenAI’s next-generation language model training suggests AI progress may be slowing in raw “intelligence” gains—at least compared with the dramatic jumps seen between earlier generations. The claim centers on an early-stage successor to GPT-4 (often referred to as “Orion” in the reporting): after only about 20% of training, it was already said to match GPT-4-level capability on tasks and question-answering. But once training finished, the final quality improvement was described as far smaller than the step-change from GPT-3 to GPT-4, with some researchers inside OpenAI reportedly doubting it would be reliably better than its predecessor on coding.

The slowdown narrative is tied to two practical constraints: data and compute economics. Scaling up models by an order of magnitude becomes harder when the “easy” training data—broadly accessible text from the web—has largely been consumed. The reporting also points to cost pressure, quoting the view that training the next generation could run into hundreds of billions of dollars and that the scaling paradigm may eventually break. Yet the same reporting includes counterweights: other quotes from Sam Altman portray continued confidence that the path to AGI is known, that capability trajectories will keep rising for a long time, and that major breakthroughs are still possible—even if details can’t be shared.

To test whether the “plateau” idea holds up, the discussion pivots from leaks to benchmarks, focusing on Frontier Math, a set of roughly 100 difficult, unpublished problems created with input from about 60 mathematicians and writers of International Mathematical Olympiad questions. Results cited from the paper indicate current leading language models solve only about 1–2% of these problems—far below what would be expected if models were close to mastering the kind of multi-step reasoning needed for new proofs. The analysis also notes measurement uncertainty: the benchmark’s own estimated error rate is around 1%, while other benchmarks have been criticized for higher error rates (around 10%).

Even so, the conversation doesn’t land on “everything is stuck.” A key alternative explanation is data efficiency rather than brute-force scaling. Frontier Math is hard partly because relevant training material is scarce—only a small number of papers contain the kinds of reasoning steps needed. The argument is that future gains could come from better extraction of useful reasoning at inference time (test-time compute), potentially allowing models to “select” the right answer among many candidate outputs. The discussion links this to OpenAI’s “o1” family and suggests that incremental improvements in base model reasoning could still translate into better outcomes when models generate thousands of possible answers.

Finally, the pace of progress may differ by modality. Runway’s co-founder and CEO is cited saying OpenAI plans to release Sora in about two weeks, and the reasoning is that video and image domains have far more abundant training data than text. The broader takeaway: the evidence points to uneven progress—slower gains in some reasoning benchmarks, but continued momentum driven by new paradigms, better data use, and modality-specific advantages—while even insiders admit the timeline for any long-term plateau remains unknown.