AI Won't Be AGI, Until It Can At Least Do This (plus 6 key ways LLMs are being upgraded)

Current AI systems fall short of AGI largely because they struggle with genuinely novel abstract reasoning: when a task pattern hasn’t appeared in training, models often can’t generalize, even if they can sometimes recall familiar “reasoning procedures” from prior examples. That gap matters because it reframes today’s breakthroughs as impressive pattern-matching and partial program retrieval—not reliable, on-the-fly problem solving across unfamiliar situations.

A concrete example centers on an abstract reasoning challenge tied to the ARC-style test. The model behavior described is stark: it can’t reliably notice and interpret a grid transformation when the specific configuration wasn’t in its training set. The failure isn’t treated as a minor glitch; it’s presented as evidence that current large language models (LLMs) aren’t “generally intelligent” in the human sense. Instead, they often succeed by retrieving reasoning chains or programs they have effectively seen before. When the needed procedure is missing, the system may produce plausible-sounding but wrong outputs—an issue that shows up not only in benchmark reasoning but also in real-world claims.

From there, the discussion widens into a broader critique of the AI landscape: overpromising and underdelivering, persistent hallucinations, and marketing that outpaces measurable capability. Examples include rolled-back features after error-prone performance, and admissions that even consumer-facing systems can hallucinate. Privacy risks also enter the picture, with concerns about systems that continuously analyze user data such as desktop screenshots. Another major worry is “AI slop”: tools that imitate writing styles can flood platforms with low-quality, high-volume content, making it harder to trust what people read, hear, or share—especially as bots and deepfakes scale.

Yet the narrative refuses a binary “AI is hype” versus “AGI is imminent” framing. The core counterpoint is that the reasoning limitations aren’t necessarily a dead end. Multiple research directions aim to make LLMs more capable at the kind of generalization ARC demands, without relying on naive scaling alone.

Six pathways are highlighted. First is compositional generalization—training models to combine learned reasoning blocks into new structures, with evidence from small transformer models that can mimic aspects of systematic generalization. Second is better program discovery using verifiers and search: approaches that train reward models to flag faulty reasoning steps, or use external simulators to evaluate many candidate solutions. Third is active inference via test-time fine-tuning, where models adapt during evaluation using a small set of examples plus synthetic data to learn the task’s structure on the fly. Fourth is hybrid planning that pairs LLMs with symbolic systems, letting the language model generate candidate plans while a symbolic checker validates and refines them. Fifth is algorithmic grounding through specialized neural components (e.g., graph neural networks) whose learned algorithms are then made usable by language models through embeddings. Sixth is “tacit data” transfer—capturing the unprinted, conversational, and failure-driven knowledge that humans accumulate, especially in mathematics.

The takeaway is pragmatic: AGI isn’t shown as imminent, but progress toward more general reasoning is portrayed as achievable through combinations of training strategies, external verification, adaptive inference, and tighter coupling between language models and tools that can check or execute plans. The most realistic near-term path, the discussion suggests, is not a single leap but a convergence of methods that reduce the reliance on memorized procedures and increase the ability to synthesize new ones when the situation is unfamiliar.