Build AI Agent from 0 to Production Deployment | LangChain, Ollama, MLflow & Docker (Full Tutorial)

A unit-conversion AI agent can be built end-to-end—from a single-tool loop to a streaming REST API—then packaged into a Docker container and deployed to production. The practical payoff is a working service where one request like “convert 10 mi to kilometers and 150 lbs to kilograms” triggers multiple tool calls (distance and weight) and returns the final converted values in a single response.

The build starts with a clear definition of an agent: a language model (LM) that has access to an environment plus a set of callable tools. In this setup, the LM receives the user’s question (optionally with chat history) and decides whether it should stop or call one or more tools. Tool outputs are fed back into the LM, which then produces the final answer. The agent’s control flow is implemented as a loop that keeps running until the LM indicates completion.

The first milestone is a rudimentary “single tool” agent for weight conversion. A conversion tool is defined as a LangChain tool that takes inputs like a numeric value plus source and target units, then performs the conversion using standard factors (for example, 1 kg ≈ 2.2 lb). A system prompt instructs the LM to use the provided tools to complete the task. During execution, the LM emits a tool call, the tool returns the computed result, and the LM formats the final response.

Next comes an upgrade: streaming output and a safer execution loop. The agent is modified to accept user input dynamically, stream the LM’s response chunks to the client, and capture tool-call events (tool name, arguments, and tool result). A max-iteration limit prevents the loop from running indefinitely, and the agent raises an error if it hits that ceiling.

The agent then gains more capability by adding additional tools—specifically distance and temperature conversion—so it can handle multi-part queries. With multiple tools registered, the LM can call them sequentially within the same request, producing combined results such as converting kilograms to pounds and kilometers to miles.

To make the system production-ready, observability is added using MLflow. The setup configures an MLflow tracking URI, creates an experiment (e.g., “unit conversion agent”), and records traces that show the tool list, system prompt, tool calls, tool outputs, and the final agent response. This makes it possible to inspect what happened “under the hood” during development and after deployment.

Finally, the agent is wrapped in a FastAPI service. An asynchronous “ask” endpoint streams results back to clients using Server-Sent Events (text/event-stream). A response generator converts the agent’s content chunks and tool execution records into JSON payloads so clients can render both the evolving text and the underlying tool activity. Environment variables control the model name and provider so local development can use Ollama while production swaps to Gemini 2.5 Flash.

Packaging and deployment complete the pipeline. A multi-stage Docker build reduces the shipped image size by separating dependency compilation from runtime. Docker Compose runs the container locally with host access settings, health checks hit the root endpoint every 30 seconds, and the same container is deployed to Render as a Docker web service. In production, the API behaves the same but uses Gemini 2.5 Flash, delivering faster responses while preserving the streaming interface and tool-call transparency.