0
When building conversational assistants with Large Language Models, keeping track of conversation history without exceeding token context limits is a common challenge. Standard buffer memory simply appends every user and assistant message to the prompt, which quickly exhausts the context window or inflates API costs.
I am working on a customer support bot where conversations can last dozens of turns. I want to know how developers implement robust conversation memory in LangChain when dealing with long user sessions.
Here is my current basic setup using Python and LangChain summary buffer memory:
from langchain_openai import ChatOpenAI
from langchain.memory import ConversationSummaryBufferMemory
# Initialize the LLM with low temperature for deterministic context summarizing
llm = ChatOpenAI(model="gpt-4o-mini", temperature=0.0)
# Set up summary buffer memory with a strict token threshold
memory = ConversationSummaryBufferMemory(
llm=llm,
matoken_limit=300 # Summarizes older turns when context exceeds 300 tokens
)
# Simulate adding interactions to the conversation history
memory.save_context(
{"input": "Our deployment failed on stage 2 with a database connection timeout."},
{"output": "Please verify the PostgreSQL connection string and subnet security groups."}
)
# Retrieve current context formatted for prompt injection
current_history = memory.load_memory_variables({})
print(current_history["history"])
