Grok-2 Actually Out, But What If It Were 10,000x the Size?

Grok 2 is now available for testing through a Twitter chatbot, but the bigger story isn’t just how it benchmarks—it’s what its release signals about where large language models may be heading: toward internal “world models” that let them reason about cause and effect, not merely pattern-match text. With no paper or model card published yet, the only concrete evidence comes from benchmark results and hands-on testing, which suggest Grok 2 is strong, but still not the top performer across the board.

On standard LLM evaluations, Grok 2 lands near the very top. In the Google Proof Science Q&A Benchmark and the MLU Pro think of it as a subject-knowledge test with fewer distractions, Grok 2 places second, behind Claude 3.5 Sonic. On at least one math benchmark, it scores highest in that set. The transcript also notes a separate “Simple bench” effort used for reasoning checks; without Grok 2 API access, testing relied on a custom question set. That informal run still produced answers that often look well-reasoned—scoring over 90% on the author’s reasoning-style questions—but it also misses items that Claude 3.5 Sonic gets right, implying Grok 2 may lag on that particular reasoning profile.

A key practical detail is that Grok 2’s system prompt appears to have been inferred from a jailbreak. The prompt draws inspiration from The Hitchhiker’s Guide to the Galaxy and includes constraints about not having access to internal x/Twitter data and systems, with an overall goal of maximizing truthfulness. That matters because the transcript frames the next bottleneck for society not as raw intelligence alone, but trust: as image and video generation improves, fake media will spread faster than verification can keep up. The discussion points to near-term realism—possibly within months to a couple of years—where even video calls could become hard to trust.

To counter that erosion of shared reality, the transcript argues that technical solutions like zero-knowledge proofs could enable “personhood credentials” without relying on easily forged identifiers. It also highlights a parallel challenge: tracing the provenance of synthetic outputs back to training sources. A Google executive reportedly discussed methods to trace outputs to training data fractions for creator compensation, but the transcript calls that approach nearly impossible for highly creative outputs.

Finally, the transcript widens from Grok 2 to scaling forecasts. An Epoch AI paper is cited for projecting how much compute and model scale could grow by 2030, with the headline claim that models might reach roughly 10,000× the scale of GPT 4. The transcript treats that as plausible given constraints like data scarcity, chip production, and power—while emphasizing that scale alone may not guarantee a step-change. The more important question is whether models are learning latent causal structure—internal simulations of how the world works. Experiments described in the transcript suggest language models can develop hidden concepts and even track states across puzzles, with training sometimes producing correct “instructions” at high rates. The open question remains whether future systems will become AGI directly, or whether they’ll mainly serve as interfaces that drive separate world simulators—or, alternatively, become engines for convincing deepfakes.