OpenAI GPT-4o | First Impressions and Some Testing + API

OpenAI’s newly released GPT-4o models are positioned as a real-time, multimodal “reasoning” system that can work across text, images, and audio with notably low latency—reported around 320 milliseconds, roughly in the range of typical human conversational turn-taking. That speed matters because it makes interactive voice and vision use feel less like a chat window and more like a responsive assistant. The update also claims major cost and performance improvements, including “50% cheaper” API pricing versus GPT-4o’s prior baseline and stronger understanding for vision and audio.

Early testing described in the transcript focuses heavily on what’s available right now through the API: text and image inputs with text outputs. Audio input/output is mentioned as not yet supported in the API documentation at the time of testing, even though the live stream demonstrations included voice features such as interruption handling, real-time tone/emotion adjustments, and voice input cues (for example, responding differently when a user speaks in a “sad” tone). The tester therefore builds scripts around image analysis first, using base64 encoding and batching multiple images from a folder into a single “image analyzer” workflow.

In one image workflow, multiple slide images—each representing different “architectures” in a mixture-of-models setup—are fed into GPT-4o for description and explanation. The output is treated as strong because it produces structured summaries for each image and then synthesizes a final explanation that ties the architectures together. The transcript highlights a “mixture of models” framing: responses generated by different model roles (described as “king,” “co-founder,” and “democracy” components) are refined, discussed, and voted on to produce a more well-rounded answer.

The testing then shifts to direct image-based reasoning. A drawn triangle image is used to ask for calculations, with GPT-4o reportedly performing checks like verifying the triangle inequality theorem, determining whether it’s a right triangle via the Pythagorean theorem, and computing area. The tester also compares speed against GPT-4 Turbo using a longer writing task (three paragraphs about life in Paris in the 1800s), reporting a large gap in throughput: GPT-4o around 110 tokens per second versus GPT-4 Turbo around 20 tokens per second—described as roughly five times faster—along with lower latency and fewer tokens.

Finally, a couple of logic-style prompts are used as quick sanity checks. For a marble-in-a-microwave puzzle, GPT-4o is reported to give an answer that matches the expected outcome (the marble ends up on the microwave tray/floor area rather than inside the cup as originally oriented). Another prompt—writing sentences ending with “apples”—is used to compare accuracy, with GPT-4 Turbo reportedly hitting 10/10 while GPT-4o is said to miss one.

A major theme is access: the transcript claims OpenAI plans to bring GPT-4o to free users, which would change the competitive landscape against other assistants and multimodal models. The tester closes by promising a deeper follow-up on Wednesday, after more practical evaluation—especially once audio capabilities become testable via the API.