o1 - What is Going On? Why o1 is a 3rd Paradigm of Model + 10 Things You Might Not Know

OpenAI’s o1 preview is being framed as a third major training paradigm for large language models: not just producing fluent text or aligning outputs with “helpful and harmless” goals, but actively rewarding answers that are objectively correct—especially when correctness comes from the model’s own multi-step reasoning. The shift matters because it changes what models are optimized for. Instead of relying primarily on next-word prediction and post-hoc alignment, o1-style training pushes systems to generate reasoning traces, then uses reinforcement learning to keep only the reasoning paths that lead to correct results.

At the core is a training loop that combines “test-time compute” (letting the model spend more steps thinking before answering) with “train-time compute” (fine-tuning on the best reasoning generations). The transcript describes an approach where models first generate many candidate chains of thought at higher diversity settings, then a grading mechanism filters them. The key is data efficiency: rather than learning from noisy web text that may include incorrect or irrelevant content, the process uses “golden data” consisting of correct answers paired with correct reasoning steps. That filtering is what turns reasoning into a scalable optimization target.

A major reason this is portrayed as a step change is that it addresses a long-standing weakness of chain-of-thought prompting: reasoning can be wrong even when it looks plausible. The proposed fix is to train the model with reinforcement learning so that incorrect reasoning is less likely to survive selection. The transcript emphasizes that o1’s gains show up most clearly in domains where correctness can be unambiguously checked—math, science, and coding—while performance can lag or regress in areas where “correct vs. incorrect” is harder to define, such as personal writing.

To make the practical impact concrete, the transcript uses a “librarian” metaphor. Earlier systems are likened to librarians who retrieve the right book but point to the wrong paragraph; o1-style systems are treated more like librarians who have learned which specific parts of books reliably answer questions. Still, the metaphor leaves an important limitation: if a question falls outside the training distribution—if the needed facts or procedures aren’t in the library—then even a better librarian can fail. The transcript also argues that there is no foundation model for the physical world yet, which helps explain why models can struggle with real-world spatial or physical reasoning.

Several “hidden hints” are offered about how o1 might work under the hood. The transcript connects o1 to earlier research such as “Let’s Verify Step by Step,” describing a verifier or reward model that checks individual reasoning steps to reduce false positives where the final answer is correct for the wrong reasons. It also draws parallels to chess: systems like Stockfish improved when evaluation moved from handcrafted heuristics to neural networks that effectively learned better internal “reasoning” strategies. The overall forecast is that as long as there’s a reliable grader for correctness, performance should keep rising with more compute and better verification.

Finally, the transcript notes that government interest is growing. The White House is described as taking these developments seriously, citing projects like o1 and Strawberry as relevant to national security and economic interests. The open question remains whether these advances amount to humanlike intelligence or general intelligence; the transcript’s stance is that o1 is a strong leap for narrow, checkable tasks, but it is not yet a complete solution for AGI.