Never Browse Alone? Gemini 2 Live and ChatGPT Vision

New multimodal “sidekick” tools from Google and OpenAI are moving from one-off image or text answers to live, interactive experiences—sometimes even with web browsing and multi-step actions. The most eye-catching capability is Google’s Gemini 2.0 in AI Studio, where a user can grant camera access and hold a real-time conversation about what’s in front of them. That convenience is paired with a warning that accuracy can wobble: when asked to verify a highlighted leaderboard claim, Gemini 2.0 Flash initially agreed, then corrected itself after checking the ranking, underscoring how easily confidence can outpace verification.

Google also introduced “Deep research” inside Gemini Advanced, positioned as a web-research assistant that compiles results from roughly 20 sources and produces a synthesized plan. In practice, the transcript shows how such outputs can be comprehensive yet still unreliable on specific factual details—an example sentence about benchmark performance was wrong, and the follow-up interactive check exposed the mismatch between what was written and what the leaderboard actually showed.

The benchmark thread runs through the announcements. Gemini 2.0 Flash scored around 20% on Simple Bench, a test aimed at basic reasoning (“the question behind the question”). The transcript notes additional entries such as Llama 3.3 at 19.9% and an “experimental” model at 31.1%. Google’s choice of Gemini 2.0 Flash for certain tools is framed as a pragmatic tradeoff: it’s cheaper and faster, even if it isn’t the strongest model in every benchmark. There’s also skepticism about comparisons—Google reportedly highlights results against its own models and benchmark selections that favor its narrative.

Beyond chat and vision, the most consequential shift is toward agentic computer use. Google demoed Project Mariner, where Gemini can take over a computer to click, browse, and perform actions—potentially including shopping—while staying under user control. In the transcript, Web Voyager-style testing is used to quantify progress: a “scaffold” agent reaches 67%, while Gemini 2.0 Flash-powered Mariner is cited at 90.5%. The broader point is that these systems are getting better at navigating real websites with visual inputs, not just generating text.

The conversation then widens to whether progress is slowing. Google’s CEO suggests the “low-hanging fruit is gone” and the curve is steeper, while OpenAI’s Sam Altman counters that there’s “no wall,” predicting continued improvement in reasoning and reliable multi-step action. Anthropics’ Dario Amodei is referenced with a more aggressive timeline for transformative automation.

On the OpenAI side, ChatGPT is integrated into iPhone 16 via Apple Intelligence, including “ChatGPT Vision.” The transcript stresses that live video-level interaction still requires OpenAI’s Advanced Voice mode subscription tiers (Plus or Pro), and that the vision experience is currently limited to analyzing images from within a video rather than fully live camera interaction. Overall, the central takeaway is clear: multimodal models are becoming more useful and more interactive, but accuracy and reliability—especially on factual claims and benchmark-linked statements—still demand verification, particularly as agents start acting on the web and in real-world tasks.