Bad AI Predictions: Bard Upgrade, 2 Years to AI Auto-Money, OpenAI Investigation and more

AI progress is moving faster than major forecasts from just a few years ago—especially in translation quality, image understanding, and reading comprehension—while predictions about “AI auto-money” and AGI timelines have compressed dramatically. The through-line is that tasks once labeled unsolved are now close enough to be useful, and that shift is happening on a timescale forecasters didn’t anticipate.

The update begins with a 2021 book, “A Brief History of AI,” which lists many capabilities as not yet solved and even admits uncertainty about how to make computers perform certain tasks. That pessimism is contrasted with recent demonstrations using Bard’s underlying Palm 2 model. In translation, Bard produces more human-sounding text-to-speech even for Swahili, and the claim is that Palm 2 outperforms Google Translate on quality. In multimodal reasoning, Bard is given a meme and returns an interpretation that recognizes the image as a pizza shaped like the Death Star, connects the toppings to the reference, and even explains the humor by contrasting “death and destruction” with “food and enjoyment.” Bard also incorporates Google Lens for real-world assistance—identifying objects on walks—though it can refuse to answer when a human face appears in the frame. A more striking anecdote is that an image taken in a local park was sometimes recognized as the park’s location, even when it wasn’t widely known.

The transcript then pivots to benchmark-style evidence. A 2021 forecast for a math dataset predicted scores rising from 21 (2023) to 52 (2025), with a projected 80 only in 2028. Current performance is described as already near that mark: GPT-4 is said to reach 78% on the dataset without code interpreter or Wolfram Alpha, and experiments using GPT-4 with code interpreter push results to around 86%. The same pattern appears in language tasks. A 112-page novel generated by GPT-4 is presented as “interesting” even if not human-level, and the claim is that fine-tuning on an author’s work could bring models close to producing convincing full-length stories. Claude 2 is used to refine vocabulary in the GPT-4 novel, replacing generic phrasing with more vivid terms like “crystalline,” “ethereal,” and “inaugurable.” Reading comprehension is reinforced with a GRE verbal benchmark: GPT-4 is described as scoring at the 99th percentile, and the narrator reports personal practice results.

Finally, the transcript tackles the biggest forecast whiplash: AGI timelines. In 2021, predictions placed AGI in the late 2030s or early 2040s, but now estimates are pulled forward to 2026. Mustafa Suleiman of Inflection AI is cited as pushing even further, suggesting an AI could be built to make one million dollars in as little as two years—assuming strategy, product research, and some human approval. That money-making scenario is framed as potentially transformative for the global economy.

Regulation and deployment risk enter via an FTC investigation into OpenAI, described as a detailed document focused on internal communications about hallucinations and privacy risks. The transcript suggests that if penalties follow, companies may become more cautious about publicly releasing models. It closes with the competitive acceleration of funding—Elon Musk’s xAI offering up to $200 million in signing bonuses—and a nod to iRobot’s question of whether machines can create art, now with a more optimistic ending.