Building a LangChain Custom Medical Agent with Memory

A LangChain “medical advice” agent can be built to answer questions using a site-restricted search (WebMD) and to carry context across multiple turns via conversation memory. The key move is combining a custom tool wrapper for “site:WebMD” queries with a custom prompt template and a custom output parser, so the model can reliably decide when to search, how to feed the query into that search, and when to stop and produce a final answer.

The system starts with a single search tool built on DuckDuckGo, but the tool is modified to constrain results to WebMD. Instead of letting search results come from any site, the wrapper prepends “site:WebMD.com” to the user’s query. That turns a generic search into a targeted retrieval step, making the agent’s downstream answers traceable to one source domain.

Next comes the agent’s prompt and scratchpad handling. The prompt instructs the model to answer as a compassionate medical professional while using the available tool(s). A custom prompt template is used to manage intermediate reasoning steps in a React-style loop: the scratchpad begins empty, then accumulates the model’s “thoughts,” “actions,” and “observations” as it iterates. Importantly, the scratchpad and tool wiring are generated dynamically by the agent framework rather than being manually supplied by the user.

Reliability hinges on the custom output parser. The parser inspects model output for a “final answer” marker; if present, it signals the agent to finish. If not, it uses regex to extract the requested tool name (“action”) and the tool input. When the output doesn’t match the expected format—such as missing an action or failing to include parseable fields—the parser returns an error like “could not parse LLM output,” a practical reminder that many open-source models may not always follow strict tool-call formatting.

The agent is then assembled into an AgentExecutor with stopping behavior tuned to the “observation” boundary. In debug mode, the workflow becomes visible: the model emits an action like “search WebMD” with an action input such as “sprained ankle treatment,” the tool returns search text, and the agent uses that observation to craft the final response. The example demonstrates a full cycle from tool selection to final answer generation.

Finally, conversation memory turns one-off Q&A into a multi-turn assistant. Without memory, follow-up questions like “what meds could I take?” would lack the earlier condition context. With ConversationBufferMemory configured to remember the last two turns, the agent receives prior user questions and its own earlier responses, allowing it to correctly interpret subsequent prompts as referring to the same condition (e.g., a sprained ankle). The resulting answers include practical guidance such as cold compresses and typical healing timelines (given as “4 to 21 days” in the example), along with safety language urging medical attention if symptoms persist or worsen.

Overall, the build pattern—site-restricted retrieval tool + prompt/scratchpad control + output parsing + optional memory—can be repurposed for Q&A over other trusted sources, including adding tools like Wikipedia lookups or custom APIs.