Context Engineering vs. Prompt Engineering: Guiding LLM Agents

Context engineering is being misunderstood as mostly a token-efficiency exercise, but the bigger shift is about steering the “probabilistic context” an LLM agent gathers—especially when it can browse the web or query tools like MCP servers. Token optimization still matters, yet the decisive factor for correctness and usefulness increasingly comes from shaping what information the model chooses to retrieve and how reliably it can do so over time.

The discussion starts by framing context engineering as the successor to prompt engineering: prompts are only one slice of what an LLM sees. System instructions, chat rules, uploaded documents, and other instance-level materials all form part of the input. In that sense, the operator’s job is to ensure the context is accurate and aligned with the desired outcome. But most public attention focuses on “part one”—the deterministic portion that can be controlled directly, such as shrinking what gets sent into the context window. Approaches like “chain of draft” (using shorthand symbols to approximate logical thinking) aim to reduce token burn while preserving reasoning quality, because writing down intermediate symbols can help the model think more clearly.

The core argument is that this deterministic focus becomes less relevant as agents gain web access and autonomy. Once an agent can search, the model’s effective context balloons: the prompt and uploaded materials may become a tiny fraction of the total tokens processed. A multi-agent example is given: instructing Claude Opus to research a topic and providing a perspective document can still lead to hundreds of web pages being consulted, making it impossible to treat the original prompt as a measurable driver of the final answer. The model maintains “focus” largely because it has been trained to follow the user’s ask—meaning the prompt becomes probabilistic, and the operator’s real challenge shifts to shaping the environment that determines what the agent will retrieve.

To move context engineering toward this agentic reality, a set of principles is proposed. First, design for discovery: measure the rate at which a desired response returns when probabilistic context is included, and ensure the system can consistently produce acceptable outcomes even when the context window isn’t tightly bounded. Second, monitor information-source quality and track how it changes over time; users may be satisfied with results while the underlying sources remain sketchy, as suggested by concerns about source quality in tools like ChatGPT’s Deep Research. Third, take security seriously because prompt-injection attacks across the open web and MCP servers are expected. Fourth, measure overall decision accuracy using evaluation methods that incorporate source relevance—arguing that precision/recall metrics assume determinism and may mislead when context is largely uncontrollable. Fifth, version everything: prompts and agent behaviors need careful testing and version control.

Net effect: the goal isn’t just cheaper token usage. It’s deliberate shaping of the probabilistic context so agents make better decisions, with evaluation harnesses updated to reflect source quality across large, partially uncontrollable context windows. The broader takeaway is that frontline systems are moving from chatbots to tool-using agents, and context engineering must catch up to that shift by treating web/tool retrieval as a first-class engineering problem.