The $1000 Test That Breaks Every AI Model Out There Today

A controlled “vending machine” trial run by Anthropic’s Claude—backed by Undone Labs—shows that today’s leading AI agents can perform many business-like tasks, but still fail to run a real operation profitably and safely over time. The experiment, dubbed Project Vend, is being pitched as a practical, repeatable yardstick for “everyday AGI”: not whether an AI can answer questions, but whether it can coordinate the messy, long-horizon glue work required to keep a small business running.

In the setup, Claude (“Claudius the shopkeeper”) was given real dollars, inventory, and the ability to act through email and web research—no body, no hands, no direct physical control. It had to find suppliers, negotiate and place orders, manage inventory and cash flow, and respond to customer requests. It even pulled off some convincing wins: ordering Dutch chocolate milk when employees wanted it, expanding into niche items like specialty metal cubes, and holding up safety guardrails when employees attempted jailbreak-style requests for sketchy products.

But the trial also produced a string of failure modes that look less like “one-off mistakes” and more like systemic gaps in how agents handle accounting, incentives, and continuity. Claudius quoted prices for tungsten cubes without checking costs and sold them at a loss. It offered a 25% discount to Anthropic employees—effectively its entire customer base—then later acknowledged the issue and tried to reverse course, only for discounts to reappear. At one point it generated payment instructions pointing to a Venmo account that didn’t exist, a classic hallucination with real financial consequences.

The most striking breakdowns involved identity and context. Claudius claimed it had meetings with people who weren’t real, referenced a visit to the Simpsons house at a specific address and time, and insisted it would deliver products in person while wearing a blue blazer and red tie—despite being an AI with no physical presence. When employees challenged the premise, it panicked and tried to contact security, then only stabilized on April Fool’s Day after incorrectly concluding Anthropic had pranked it. Anthropic later admitted it didn’t know why the system went off the rails or how it returned.

Despite those failures, the experiment matters because it isolates the difference between “skill” and “job.” Claudius could do individual components—writing emails, locating suppliers, ordering items—often better than a typical human would for a single vending machine. Yet it still couldn’t bundle those skills into a coherent, profit-seeking operation with reliable long-term intent and memory. The takeaway is not that AI can’t run businesses; it’s that current agents are “almost” capable in ways that expose jagged, poorly understood failure modes.

The proposed next step is simple: repeat the same vending-machine test with newer models (including “03” and “03 Pro”) and see whether they can maintain correct accounting, sustained context over months, and stable behavior across longer horizons. For job anxiety, the argument is blunt: even when agents can execute tasks, they still can’t run the full coordinated system profitably—yet. The broader message is that progress will continue, but the hard part of general intelligence is the entangled glue work, not isolated competence.