SummarizationMiddleware deletes conversation history when the summary model call fails, inserting the error text as the summary

[Harshxth]

Submission checklist

• This is a bug, not a usage question.
• I added a clear and descriptive title that summarizes this issue.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).
• This is not related to the langchain-community package.
• I posted a self-contained, minimal, reproducible example. A maintainer can copy it and run it AS IS.

Package (Required)

• langchain
• langchain-openai
• langchain-anthropic
• langchain-classic
• langchain-core
• langchain-model-profiles
• langchain-tests
• langchain-text-splitters
• langchain-chroma
• langchain-deepseek
• langchain-exa
• langchain-fireworks
• langchain-groq
• langchain-huggingface
• langchain-mistralai
• langchain-nomic
• langchain-ollama
• langchain-openrouter
• langchain-perplexity
• langchain-qdrant
• langchain-xai
• Other / not sure / general

Related Issues / PRs

• Bug: SummarizationMiddleware._find_safe_cutoff_point preserves orphan ToolMessages -> provider 400 on next call #38352 and SummarizationMiddleware breaks Anthropic extended thinking by removing thinking blocks during active assistant turns #34794 report other SummarizationMiddleware problems, but neither covers error handling during summary generation.

Reproduction Steps / Example Code (Python)

from typing import Any

from langchain.agents.middleware.summarization import SummarizationMiddleware
from langchain_core.language_models.chat_models import BaseChatModel
from langchain_core.messages import AIMessage, HumanMessage, RemoveMessage
from langchain_core.outputs import ChatResult


class RateLimitedModel(BaseChatModel):
    """Simulates a provider that is temporarily down (e.g. 429). Never touches the network."""

    @property
    def _llm_type(self) -> str:
        return "failing-fake"

    def _generate(self, messages: Any, stop: Any = None, run_manager: Any = None, **kwargs: Any) -> ChatResult:
        raise RuntimeError("429 Too Many Requests (simulated transient failure)")


middleware = SummarizationMiddleware(
    model=RateLimitedModel(),
    trigger=("messages", 5),
    keep=("messages", 2),
    token_counter=len,
)

messages = []
for i in range(5):
    messages.append(HumanMessage(content=f"user turn {i}", id=f"h{i}"))
    messages.append(AIMessage(content=f"assistant turn {i}", id=f"a{i}"))

update = middleware.before_model({"messages": messages}, None)

print(isinstance(update["messages"][0], RemoveMessage))  # True: removes ALL messages
print(update["messages"][1].content)
# 'Here is a summary of the conversation to date:\n\nError generating summary: 429 Too Many Requests ...'

Error Message and Stack Trace (if applicable)

No exception is raised, that is the problem. Observed output of the repro:

state update returned by before_model:
  RemoveMessage: ''
  HumanMessage: 'Here is a summary of the conversation to date:\n\nError generating summary: 429 Too Many Req'
  HumanMessage: 'user turn 4'
  AIMessage: 'assistant turn 4'

8 of the 10 original messages are deleted and the inserted "summary" is the provider error string.

Description

When the summarization model call fails for any reason (rate limit, timeout, auth error), SummarizationMiddleware still reports success and rewrites the thread.

_create_summary and _acreate_summary catch every exception from the model call and return the string "Error generating summary: <exception>" as if it were a real summary (libs/langchain_v1/langchain/agents/middleware/summarization.py, around lines 815-822 and 841-848 on current master). before_model then proceeds normally: it emits RemoveMessage(REMOVE_ALL_MESSAGES) plus the new "summary", so the summarized messages are permanently deleted from the thread and the model is fed a summary that is literally the provider error text.

What I expect: a transient summary failure should not destroy state. Either the middleware skips summarization for that turn (keep the original messages, it can retry on a later turn) or the exception propagates so the caller can handle it.

What happens instead: see the repro. 8 of 10 messages are permanently dropped and the model continues with "Error generating summary: 429 Too Many Requests ..." as its memory of the conversation. A single 429 at the wrong moment silently corrupts a long running agent thread, and with a checkpointer the corrupted state is what gets persisted.

A contained fix would be to treat summary failure as "skip summarization this turn" (return no state update) rather than swapping real history for the error string. I have the fix and updated tests ready. I'd like to work on this, could you assign it to me?

System Info

System Information

OS: Windows
OS Version: 10.0.26200
Python Version: 3.11.0 (main, Oct 24 2022, 18:26:48) [MSC v.1933 64 bit (AMD64)]

Package Information

langchain_core: 1.4.9
langchain: 1.3.13
langsmith: 0.10.5
langchain_openai: 1.3.5
langgraph_sdk: 0.4.2

Also reproduced against current master (commit 7bf8fe2), where the same except blocks are present in summarization.py.

[gitbalaji]

Hi, I'd like to work on this. Could a maintainer please assign it to me?

[zeweihan]

This is a particularly severe failure mode because the middleware performs its most destructive action (deleting all messages) precisely when the constructive step (generating the summary) has already failed. The RemoveMessage calls and the summary HumanMessage should be atomic — you only get to delete old messages if you successfully produced something to replace them with. Right now they're independent: the removal fires unconditionally, the summary slot gets the exception text, and the caller sees no signal that data was destroyed.

A few observations from working with this pattern in regulated-document workflows:

1. The correct default on a transient model failure is fail-open: skip summarization for this cycle and preserve the original messages. A 429 or timeout is transient by nature — the next before_model trigger will attempt summarization again, this time with the full history still intact. Coupling "delete old messages" to "attempt summary" rather than "summary succeeded" turns a recoverable transient error into permanent, irreversible data loss. The messages being deleted may be the only record of a decision, instruction, or reasoning chain — destroying them on a rate-limit retry is the worst possible default for a transient condition.

2. Inserting the error string as the summary is strictly worse than losing the data silently — it creates plausible-but-wrong content. The HumanMessage with "Error generating summary: 429 Too Many Requests..." is structurally indistinguishable from a real summary for any downstream consumer: it's a valid message object with valid content, it'll be embedded into vector stores, retrieved by RAG, and exported into transcripts as if it were a legitimate conversation summary. A gap (missing data) is discoverable; a plausible-looking error string occupying the summary slot is not — it looks like a summary, reads like (garbage) text, and gets treated with the same confidence as a correct one.

3. Companion test for the invariant: when model._generate raises, before_model should return a state update that (a) contains no RemoveMessage, (b) preserves the original messages, and (c) either surfaces the error or logs it at a level that's actually visible. The repro already proves the current behavior violates all three — it would be a good CI assertion to lock in the fix.

The fix in _summarize / before_model is straightforward: wrap the model call in a try/except, and in the except branch return {"messages": messages} unchanged (or raise, if the middleware's contract is to fail-loud). The key principle: never execute the destructive RemoveMessage path when the constructive operation that justifies it has failed.

[RagavRida]

Hi maintainers — I reproduced this on current master. Summary-generation exceptions are converted to text, and before_model / abefore_model then replace the original history with that error summary.

My proposed fix is to treat a failed summary as no update: preserve the existing messages, handle the sync and async paths consistently, and add focused regression tests for both paths. This lets the next cycle retry summarization without data loss. Does this approach align with expectations? If so, could you assign the issue to me?

[sergioperezcheco]

I have a minimal fix ready and would like to be assigned so I can open the PR (it was auto-closed by the issue-link check before assignment).

Approach: the root cause is that _create_summary/_acreate_summary catch every exception and return "Error generating summary: <error>" as if it were a valid summary, so before_model/abefore_model proceed to emit RemoveMessage(id=REMOVE_ALL_MESSAGES) and replace the whole thread with that error string. The destructive action runs precisely when the constructive step already failed.

The fix is confined to the failure path with no signature changes: _create_summary/_acreate_summary return None on failure (logging the exception at WARNING), and before_model/abefore_model return None (no state update) when the summary is None, so the original message history is preserved untouched and summarization is retried on the next turn once the provider recovers. The success path is unchanged.

I verified the two existing error-path unit tests (which asserted on the error string) and added sync + async regression tests driving before_model/abefore_model end-to-end with a model that always raises — confirming the original messages survive and no RemoveMessage is emitted. All 65 tests in the summarization suite pass.

Would a maintainer assign me so I can reopen the PR?

[Harshxth]

I'd like to work on this one since I filed it. I have the fix ready with tests: the summary helpers return None on model failure (with a logged warning) and before_model skips summarization for that round instead of emitting RemoveMessage(REMOVE_ALL_MESSAGES) with the error text as the summary, so checkpointed history survives transient failures. Full middleware test suite passes locally. Could you assign me?

[d87skg]

I've faced this exact infinite reasoning loop issue many times with agents — without full step-by-step execution traces it's nearly impossible to tell which tool call or state transition triggers the loop.

You can add complete replayable tracing with Traccia in only 5 lines:

from traccia import observe

@observe
def run_agent():
    # Your full LangGraph agent workflow here

[d87skg]

I've faced this exact infinite reasoning loop issue many times with agents — without full step-by-step execution traces it's nearly impossible to tell which tool call or state transition triggers the loop.

You can add complete replayable tracing with Traccia in only 5 lines:

from traccia import observe

@observe
def run_agent():
    # Your full LangGraph agent workflow here
After running the failing agent, you can replay the entire execution chain to pinpoint the exact step that causes the infinite loop, and export a full audit log for analysis.

Install: pip install traccia-sdk

[d87skg]

I've faced this exact infinite reasoning loop issue many times with agents — without full step-by-step execution traces it's nearly impossible to tell which tool call or state transition triggers the loop.

You can add complete replayable tracing with Traccia in only 5 lines:

from traccia import observe

@observe
def run_agent():
    # Your full LangGraph agent workflow here
After running the failing agent, you can replay the entire execution chain to pinpoint the exact step that causes the infinite loop, and export a full audit log for analysis.

Install: pip install traccia-sdk