Build Anything with Local Agents, Here’s How

Running AI agents locally—without paying API fees—hinges on two pieces: a local model runtime (Ollama) and an agent framework (CrewAI). The setup described pairs Ollama with open-source models so chats and outputs stay on the user’s machine, then uses CrewAI to coordinate multiple specialized agents that generate system prompts and task outputs. The practical payoff is automation: instead of hand-writing system prompts for every new agent, a “prompt engineering” agent can draft structured instructions in seconds.

The walkthrough begins by installing Ollama from ama.com, launching it, and then installing CrewAI in a Python environment. It emphasizes environment hygiene (creating a dedicated virtual environment in VS Code) and then installs dependencies for agent orchestration and model integration. For model access, it uses LangChain’s Ollama integration, noting that the community package is the recommended path. From there, the guide selects a specific open-source model—Mixtral (a Mixture of Experts model)—and explains how model choice depends on hardware and speed. Mixtral’s size and resource demands are framed against typical consumer setups: smaller models like 7B are easier on memory, while larger variants (13B, 47B, 70B) require more capable machines.

Once Mixtral is running in Ollama, the code links the model into the agent framework so CrewAI can call it automatically. The next step is defining a reusable workflow driven by a variable called topic. That variable lets the same agent team generate prompt instructions for different domains—web development, customer support, programming, financial analysis, or essay writing—without rewriting everything.

Two agents are created: a research agent and a prompt engineer. Each agent receives a goal and a backstory formatted as system-style instructions, with verbosity enabled and delegation disabled to keep the workflow focused. The research agent’s job is to gather relevant information about what an expert in the chosen topic would do; the prompt engineer then turns that understanding into a single structured system prompt written in Markdown. Both agents are paired with tasks that specify descriptions and “expected output” examples to improve consistency.

A CrewAI “crew” ties the agents and tasks together in a sequential process: research first, then prompt engineering. The walkthrough includes troubleshooting points—such as using the correct parameter names (LLM vs model) and ensuring the code uses “agents” and “tasks” correctly—before showing successful runs. The resulting system prompts are presented as already strong: when switched from web development to customer support, the generated instructions include clear guidelines on policies, clarity, and minimizing confusion.

Overall, the core insight is that local agent teams can produce usable, structured system prompts quickly—often better than what many teams write manually—while keeping data private and avoiding recurring API costs. The guide positions this as a starting point, with a promise of further optimization in a follow-up.