How To Implement Short Term Memory Using LangGraph

Short-term memory in LangGraph isn’t something LLMs can keep on their own—so the practical fix is to store conversation state outside the model and feed it back in a controlled way. The walkthrough builds that capability step by step: first by using LangGraph’s checkpointer plus a per-conversation thread ID to maintain a “conversation buffer,” then by replacing volatile RAM storage with persistent PostgreSQL so the state survives restarts. The result is a production-shaped pattern for agentic chat where “what was said before” remains available across turns and across process lifecycles.

The core starting point is the stateless nature of LLM calls: each invocation behaves like a fresh conversation unless prior messages are explicitly provided. To simulate short-term memory, the guide maintains a running message history and appends it to each new LLM call. In LangGraph terms, that state is stored at every superstep via a checkpointer. A thread ID (e.g., “thread one” vs “thread two”) determines which conversation history gets loaded and updated. With this setup, asking “What is my name?” after earlier messages correctly returns the previously provided name—because the same thread’s stored messages are reattached to the next model call.

That works until the system restarts. The checkpointer initially stores state in RAM, which disappears when the program exits. After a restart, the guide demonstrates that the prior thread’s messages are gone, and the model can no longer answer based on earlier context. This exposes the biggest gap in the naive short-term memory implementation: volatility. The fix is persistence—LangGraph’s recommended approach is to use a PostgreSQL-backed checkpointer (the walkthrough uses a Docker-based PostgreSQL setup). After wiring the checkpointer to a database URL and running the graph inside a context manager, the same thread history can be fetched after restarting the Python process, and the model continues to respond with the remembered details.

Once memory is persistent, the next bottleneck appears: context window overflow. Because the short-term approach keeps concatenating the full conversation history, long chats can exceed the LLM’s maximum token limit, leading to unreliable answers and hallucinations. To prevent that, the walkthrough introduces two mitigation techniques.

First is trimming: before calling the LLM, the system counts tokens approximately and enforces a maximum token budget (example uses 150). If the conversation history would exceed the limit, older messages are not deleted from state, but they are omitted from what gets sent to the model—only the most recent messages that fit remain in the prompt.

Second is summarization, which addresses trimming’s downside: older messages are ignored entirely, even when they contain useful information. Summarization keeps the latest messages while compressing older content into a running summary generated by the model. When the message count crosses a threshold (example triggers when more than six messages exist), the system summarizes the earlier portion, deletes those older messages from state, and retains only the summary plus the newest turns. The final flow uses a conditional graph edge: chat proceeds normally until the threshold is exceeded, then a cleanup-and-summarize node updates the summary and prunes the message list.

Together, the walkthrough delivers a complete pattern: store per-thread state with a checkpointer, persist it with PostgreSQL for restart safety, and manage prompt size with trimming and summarization to avoid context overflow while keeping important prior information available.