Build Anything with Llama 3 Agents, Here’s How

Local Llama 3 agents can be built on a modest machine, but getting reliable, fast behavior inside CrewAI may require routing the model through Gro’s API. The workflow starts with running Llama locally via Ollama (including downloading the 8B or 70B model), then wiring two CrewAI agents—one to classify an email as “important,” “casual,” or “spam,” and a second to draft a concise reply based on that classification. The practical payoff is an end-to-end “agent team” that can process an input email and produce both a label and a response.

The build begins with setup: download Ollama, install VS Code, and pull the Llama 3 model from Ollama’s model list. The first run triggers a download (about 4.7 GB for the 8B model, roughly 40 GB for the 70B), which can take around 20 minutes or several hours depending on hardware. Once Ollama is running, the transcript shifts to Python. It creates a simple main.py, installs needed packages with pip (including CrewAI), and imports Ollama through LangChain Community along with CrewAI’s Agent, Task, Crew, and Process.

Two agents are defined. The “email classifier” agent is given a role, a goal to label each email as important/casual/spam, and a short backstory; it uses the local Llama 3 model and enables verbose logging while disabling delegation. The “responder” agent follows the same pattern but focuses on writing a concise, simple response tailored to the email’s importance category. Next comes task definition: one task uses an f-string to insert the email text and expects one of the three labels, while the second task instructs the system to respond to the email. Those tasks are assembled into a Crew with sequential processing, then kicked off and printed.

A key snag appears during testing: the Llama 3 model behaves correctly when run directly in the terminal, yet CrewAI integration produces problematic behavior (including suspicion of hallucinations or slower, inconsistent outputs). The classifier still sometimes returns the expected “spam” label for a Nigerian prince-style phishing example, but the agent-run path is unreliable enough to warrant a fix.

The solution is to connect CrewAI to Gro’s API. The transcript walks through creating a Gro API key, setting environment variables (API key, API base URL, and model name), and importing OS to read those values. After removing the explicit local LLM assignment—letting CrewAI use its default configuration—the same agent workflow runs successfully through Gro, with dramatically improved speed. The result is a practical “agent team” that preserves the two-step classification-and-response logic while trading local inference for a faster, more dependable API-backed model call.