Build Hour: Agent RFT

Agent RFT (agent reinforcement fine-tuning) is positioned as a way to make tool-using agents faster and more accurate by training the model end-to-end on how to call tools and how to interpret tool outputs—using a custom reward signal that can be computed inside a customer’s environment. Instead of relying only on prompt tweaks, Agent RFT updates model weights based on learning signals that reflect “good” versus “bad” behavior during rollouts, where the agent explores many different tool-call sequences and learns from the outcomes.

The core distinction from earlier fine-tuning approaches is that Agent RFT allows the agent to interact with external tools during training. That interaction is routed through customer-provided endpoints: as the model explores, it can call tools, and each tool call is tagged with a unique rollout identifier so graders can evaluate the full trajectory—tool calls plus final answer—together. This design is meant to reduce “domain shift” between how models were trained and how customers’ real systems behave, because the agent learns using the same tools, data formats, and operational constraints it will face in production.

OpenAI’s team frames Agent RFT as a practical upgrade path: start with prompt engineering and task design (including tool descriptions and naming), then move to fine-tuning when those levers plateau. Agent RFT is presented as the next step when performance still lags—especially for reasoning-heavy agentic tasks where tool-use efficiency matters. Benefits highlighted include improved reasoning-model performance, better tool-use leading to stronger final answers, sample efficiency when training data is scarce, and lower latency. Latency improvements are attributed to two linked effects: fewer reasoning tokens and fewer tool calls, achieved through training that lightly penalizes excessive token use and can also enforce a tool-call budget.

A detailed benchmark example targets financial QA under tight constraints. The task is made harder by removing the financial report context from the prompt and requiring the agent to locate the correct report among 2,800 documents using tools, then answer within 10 tool calls. Tools include a semantic search function (built with embeddings and cosine similarity), a directory listing tool, and a “cat”-style document retrieval tool. Reward is generated with a model grader rather than brittle string matching, allowing partial credit for near-correct numeric answers and tolerance for formatting differences.

In the training demo, the baseline agent starts around 0.59 validation reward; after roughly 10 training steps, validation reward rises to about 0.63 while tool calls drop sharply. Downstream measurements show latency reductions (about 5 seconds, roughly 10%) and fewer output tokens (from about 2,500 to 1,500). Trace-level analysis reports fewer tool calls per trace (about 6.9 down to 4.2) and a shift toward trajectories that are both faster and higher-reward, with trade-offs where some samples lose reward when the policy becomes stricter.

Customer results reinforce the theme of optimizing agent behavior for real workflows. Cognition uses Agent RFT to tune planning in Devon, restricting tools to read-file and shell, and optimizing for an F1-based reward over which files the agent selects. Ambience applies Agent RFT to ICD10 coding from clinical transcripts using a code-search tool, improving F1 and reducing response time. Other examples include a GPU kernel-writing agent trained with as few as 100 PyTorch prompts (with a strong grader), and a financial reasoning workflow where a custom endpoint grader reduces hallucinations and improves core ML performance. The throughline: Agent RFT works best when tasks and grading are well-specified, tool behavior is mirrored in training, and the reward signal is hard to game while providing graded, not binary, feedback.