The Tech that’s *probably* inside GPT-5 just got Open Sourced!

Large language models don’t just get better by training bigger weights—many of the biggest gains come from “extracting” more capability out of models people already have. A viral open-source notebook tied to Matt Schumer’s work claims a smaller Claude 3 variant (Claude 3 Haiku) can be pushed close to Claude 3 Opus quality by feeding it carefully constructed examples and then generating a system prompt that makes the small model behave like the top performer. The practical pitch is straightforward: similar output quality, but far lower cost and latency, because the heavy lifting happens through prompting and example generation rather than by paying for the most expensive model on every request.

The notebook’s workflow starts with a task description plus a single input/output example given to Claude 3 Opus. From that seed, the repository generates a diverse set of additional examples, then uses the task and those examples to produce a system prompt suitable for Claude 3 Haiku (and other small or large models). It also saves the resulting system prompt and examples into a Python file formatted for generation—aimed at developers who want to drop the technique into their own products quickly. A concrete example in the transcript imagines building an AI story-writing website: instead of paying for Claude 3 Opus or GPT-4-class APIs for every user request, the approach targets Claude 3 Haiku while claiming Opus-level results, making the product cheaper to run for both the builder and end users.

That “distillation without retraining” theme is reinforced by another open-source technique called Quiet Star, attributed to Bindu Reddy. Quiet Star pushes reasoning into the generation process by having the model generate internal rationales—essentially token-by-token “inner monologues”—and then using a reward mechanism to teach which rationales lead to better outcomes. Reported results include a jump for a 7B model on common-sense question answering from 36% to 47%, alongside doubled math performance. The transcript also notes that while Quiet Star is open source and can be applied after a model is already trained (including to Claude 3 and ChatGPT), it costs more to run because it adds extra reasoning tokens.

The transcript then connects these ideas into a compounding stack: Quiet Star-style internal thinking, plus example-driven prompting that teaches a small model how to act, plus prompt-level Chain-of-Thought prompting that forces stepwise planning. The endgame is less about waiting for GPT-5 or Claude 4 to be released and more about using existing models as teachers—turning their capabilities into reusable prompting patterns and agent workflows.

Finally, the transcript points to Schumer’s open-source “Claude investor” agent, described as a constrained system that chains multiple Claude 3 calls to gather financial data, analyze sentiment and trends, and rank stocks with price targets—while explicitly warning it isn’t financial advice. The recurring message is that value can be extracted through orchestration and prompting, even when the underlying model weights remain closed. Open sourcing these prompting and agent patterns, the transcript argues, could accelerate adoption widely—because developers can implement the techniques without waiting for new frontier model releases.