GPT 4's Hidden Feature! We've been Missing Out on This!

GPT-4’s once-promised “see and explain” capability is still out there—but it’s been split across different products, with major differences in how much the models are allowed to look. The core finding from these tests: Bing Chat’s built-in vision mode can interpret images in a more human-like way than the original GPT-4 demo in some cases, yet it’s constrained by privacy and safety rules (notably face blurring). Meanwhile, an Nvidia-run vision model can be run locally and feels less restricted, but it struggles with context and fine details.

The transcript starts by revisiting OpenAI’s early GPT-4 vision demo, where the system could parse complex images panel-by-panel and even read graphs to answer math questions. That capability never landed in ChatGPT in the same form, despite file uploads. Instead, the same underlying idea appears to have been quietly integrated into Bing Chat: users can upload images directly, and the model will describe what it sees and explain jokes, objects, and scene details.

In the first comparison, Bing Chat handles a “VGA cable plugged into a smartphone charging port” meme with a generally solid explanation of the humor—absurdity from the mismatch between an old, bulky VGA connector and a modern phone port. But Bing also produces errors that don’t appear in the image, including misread or invented details about labels and connector specifics. The math/graph test shows a different failure mode: Bing appears to pull a related value from web search rather than reliably extracting the exact numbers from the chart, even though the final arithmetic result still matches.

The “ironing man on a taxi” meme produces the sharpest contrast. Bing Chat gives a longer, more structured breakdown and even references image metadata (the timestamp and location), while also making mistakes—such as confusing straps and power assumptions. Still, the transcript argues Bing’s overall interpretation is closer to what a human would do than the earlier GPT-4 response, which the tester says contained its own inaccuracies and a hallucinated “pink filter” explanation.

For the chicken nugget “world map” meme, Bing Chat delivers an explanation that aligns with the intended joke: the caption promises awe at Earth-from-space beauty, while the image is a crude, edible substitute. The transcript rates Bing’s vision-and-explanation performance around a “B minus,” attributing the gap to safety restrictions that likely reduce what the model can reliably perceive.

Finally, the transcript introduces an Nvidia vision model that can run on a home PC with high-end requirements (Windows 11+, 64GB RAM, 60GB disk, and an RTX 4090-class GPU). This model can describe faces without the same level of face blurring, but it’s more limited in language/OCR and can miss contextual relationships—like the phone still being in its packaging. The takeaway is practical: Bing is convenient but privacy-limited; the Nvidia option is more controllable but less context-accurate; and a future where AI assistants can “see” and act around the home depends on closing these reliability and safety gaps.