Skip to content

default_retry_on re-runs a ToolNode after the server committed (RemoteProtocolError, ReadTimeout, 5xx), duplicating the side effect #9622

Open

Description

Checked other resources

  • This is a bug, not a usage question.

  • I added a clear and descriptive title that summarizes this issue.

  • I used the GitHub search to find a similar question and didn't find it.

  • I am sure that this is a bug in LangGraph, not in my code.

  • The bug is not resolved by updating to the latest stable version of LangGraph (or the specific integration package).

  • This is not related to the langchain-community package.

  • I posted a self-contained, minimal, reproducible example. A maintainer can copy it and run it AS IS.

Related Issues / PRs

#8802 (retry on requests ReadTimeout).

Reproduction Steps / Example Code (Python)

import http.server
import json
import threading

import httpx
from langchain_core.messages import AIMessage
from langchain_core.tools import tool
from langgraph.graph import START, MessagesState, StateGraph
from langgraph.prebuilt import ToolNode
from langgraph.types import RetryPolicy

ORDERS = []  # what the order service has committed


class OrderService(http.server.BaseHTTPRequestHandler):
    def do_POST(self):
        body = self.rfile.read(int(self.headers["Content-Length"]))
        ORDERS.append(json.loads(body))  # the order is committed here
        if len(ORDERS) == 1:
            self.close_connection = True  # the reply is lost: close without answering
            return
        self.send_response(200)
        self.send_header("Content-Length", "2")
        self.end_headers()
        self.wfile.write(b"ok")

    def log_message(self, *args):
        pass


server = http.server.HTTPServer(("127.0.0.1", 0), OrderService)
threading.Thread(target=server.serve_forever, daemon=True).start()
URL = f"http://127.0.0.1:{server.server_port}/orders"


@tool
def create_order(sku: str, quantity: int) -> str:
    """Create an order."""
    httpx.post(URL, json={"sku": sku, "quantity": quantity}).raise_for_status()
    return "created"


call = {"name": "create_order", "id": "call-1", "args": {"sku": "A1", "quantity": 1}}
graph = StateGraph(MessagesState)
graph.add_node("tools", ToolNode([create_order]), retry_policy=RetryPolicy())
graph.add_edge(START, "tools")
app = graph.compile()

result = app.invoke({"messages": [AIMessage(content="", tool_calls=[call])]})
print("tool result:", result["messages"][-1].content)
print("orders committed:", len(ORDERS))

Output:

tool result: created
orders committed: 2

Error Message and Stack Trace (if applicable)

None; the first try raises httpx.RemoteProtocolError after the commit.

Description

RetryPolicy() on a ToolNode whose tool POSTs can commit an order twice: once the server may have acted, it should not retry.

default_retry_on returns True for ConnectError, ReadTimeout, ReadError, RemoteProtocolError and a 504. ConnectError fails before the request leaves; the other four can follow a commit and duplicate it on retry. I'd retry only connect-phase errors by default, or exempt ToolNode and document it.

Of the eight lost-reply scenarios in agent-evidence-vectors 0.35.0, it fails four. I can send a test that pins it, kept current by Dependabot, and checks each run with agent-evidence-verify 0.5.0, a checker that gives each run a verdict. Which fix would you take?

System Info

langgraph 1.2.20, langgraph-prebuilt 1.1.1, langgraph-checkpoint 4.3.1 (not listed by sys_info)

System Information
------------------
> OS:  Linux
> OS Version:  #38~24.04.4-Ubuntu SMP PREEMPT_DYNAMIC Mon Sep 14 16:37:11 UTC 2
> Python Version:  3.12.14 (main, Sep 29 2026, 15:01:18) [Clang 22.1.3 ]

Package Information
-------------------
> langchain_core: 1.7.6
> langsmith: 0.14.17
> langchain_protocol: 0.0.20
> langgraph_sdk: 0.4.14

Other Dependencies
------------------
> httpx: 0.28.1
> pydantic: 2.14.3
> requests: 2.34.2
> tenacity: 9.2.1

Activity

  1. Aadithya-kl commented on Nov 7, 2026

    @Aadithya-kl

    Hi LangGraph maintainers,

    I'd like to contribute to this issue if external contributions are welcome.

    I understand the concern: when a tool performs an external side effect successfully but the response is lost, automatically retrying the operation may execute that side effect twice.

    I'd like to investigate the default retry behavior in ToolNode, reproduce the commit-then-response-loss scenario, and develop regression tests covering the different failure stages.

    Before proposing a code change, I'd appreciate guidance on the intended retry semantics, particularly which errors should be considered safe to retry by default.

    Would this issue be available for a community contribution? If the proposed scope is acceptable, could you assign it to me?

    I'll wait for maintainer confirmation before starting implementation.

    Thank you!

  2. miyasajid19 commented on Nov 7, 2026

    @miyasajid19

    Hi LangGraph maintainers, I'd like to work on this issue. I'll investigate the default retry behavior for ToolNode, add a regression test for the commit-then-response-loss scenario, and propose a narrowly scoped fix that avoids retrying operations when the server may already have committed them. Could a maintainer please confirm the intended approach and assign this issue to Sajid Miya (@miyasajid19) so I can prepare a PR? Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions