The New Prompting Rules: How to Prompt Frontier LLM Models like Gemini 2.5, GPT 4.1 & Claude 3.7

Frontier LLMs are getting dramatically easier to use because context windows have ballooned to 200,000 tokens and beyond, letting models reliably track long conversations and ingest huge tool outputs. In practice, that means prompts can include far more source material—entire codebases, large documents, or multi-step agent traces—without the model “forgetting” key details mid-task. The transcript cites OpenAI’s guidance that long-context performance can remain strong even as accuracy gradually dips at the extreme end, and it highlights real-world experience with Gemini 2.5 where conversations starting around 120,000 tokens continued with the model still able to reference earlier information deep into the thread. The payoff is not just better recall; it also enables longer chains of internal reasoning-like behavior even for models that aren’t explicitly “reasoning” models, because more tokens remain available for intermediate work.

A second shift is that newer frontier models follow instructions more reliably, which changes how developers can design agentic workflows. Instead of hoping a model will naturally do the right thing in the right order, teams can now specify execution sequences—such as “read material first, then use it to ground the response”—and also demand ordered output formats. The transcript emphasizes that these models are especially strong at producing structured outputs in markdown, JSON, and XML. XML, in particular, has moved from a niche technique to a recommended pattern: OpenAI’s materials reportedly show XML performing well in long-context testing, and the transcript notes that XML delimiters with attributes and nested input/output examples can make complex instructions easier to constrain.

Prompting guidance also shifts toward stricter control mechanisms. Delimiters and formatting matter more as prompts grow, and the transcript suggests starting with markdown for simpler tasks, then layering XML when prompts become large or when output needs to be more repeatable. It also warns that even with better models, repetitive or highly constrained outputs may still require splitting work across multiple prompts.

Instruction wording is getting more nuanced as well. Where earlier prompting advice leaned heavily on positive examples, OpenAI’s newer guidance for GPT-4.1–class models and up reportedly allows the use of negating terms. That matters for reducing hallucinations in extraction and QA tasks: prompts can explicitly instruct the model to respond “I don’t know” when information is missing, rather than inventing details. The transcript claims Gemini 2.5 Pro behaves better on this front, but it stops short of promising zero hallucinations.

Finally, the transcript lays out a practical prompt structure: begin with role and objective, add step-by-step instructions and output formatting requirements (including schemas), include one-shot or two-shot examples plus edge cases, then provide the bulk context (customer history, device info, retrieved documents, etc.). It also recommends repeating key instructions at the end because models pay more attention to the final directives. However, it flags a cost tradeoff: repeating instructions can break prompt caching, increasing compute bills at scale.

The overall message is pragmatic: longer context and stronger instruction following make prompting more effective, but outputs still aren’t deterministic. The transcript concludes that teams should evaluate models with real metrics and expect some hallucination risk even when using advanced frontier systems.