Skip to content

ThreadWorker.close() and ThreadWorker.run_sync() block the asyncio event loop #5387

Open

Description

Submission checklist

  • This is a bug, not a usage question.
  • I added a clear and descriptive title.
  • I searched existing issues and didn't find this.
  • I can reproduce this with the latest released version.
  • I included a minimal reproducible example and steps to reproduce.

Area (Required)

  • deepagents (SDK)
  • dcode
  • talon
  • acp
  • evals
  • harbor
  • daytona
  • modal
  • quickjs
  • runloop
  • vercel
  • langsmith-sandbox
  • Other / not sure / general

Related Issues / PRs

No response

Reproduction Steps / Example Code (Python)

import asyncio
import time

from langchain_quickjs import CodeInterpreterMiddleware


# Minimal stand-ins for the deepagents ModelRequest the hooks expect.
class _Req:
    tools: list = []
    system_message = None
    def override(self, **kw):
        return self


async def main() -> None:
    # ptc=["eval"] makes awrap_model_call -> _prepare_for_call build the slot
    # via registry.get() on the cache miss. Without ptc the hook early-returns.
    mw = CodeInterpreterMiddleware(ptc=["eval"])
    req = _Req()

    async def handler(request):
        return "ok"

    async def heartbeat() -> None:
        last = time.monotonic()
        while True:
            await asyncio.sleep(0.01)
            gap = time.monotonic() - last
            last = time.monotonic()
            tag = "   <-- loop was blocked" if gap > 0.05 else ""
            print(f"heartbeat gap {gap * 1000:6.1f} ms{tag}")

    hb = asyncio.create_task(heartbeat())
    await asyncio.sleep(0.15)  # healthy baseline

    # 1) Build path: awrap_model_call -> _prepare_for_call -> registry.get()
    #    -> _build_slot_locked -> worker.run_sync()  (blocks on the loop thread)
    print(">>> await mw.awrap_model_call(req, handler)  (build: worker.run_sync)")
    await mw.awrap_model_call(req, handler)
    await asyncio.sleep(0.15)

    # Prime the worker with allocs so close()'s gc.collect() has real work
    # (an idle worker thread joins near-instantly; GC under pressure is the
    # production condition).
    thread_id = mw._fallback_thread_id
    mw._registry.get(thread_id).eval_sync(
        "let a = []; for (let i = 0; i < 2_000_000; i++) a.push({x: i}); a.length = 0;"
    )

    # 2) Evict path: aafter_agent -> registry.aevict() -> _aclose_slot
    #    -> worker.close() (thread.join + gc.collect)  (blocks on the loop thread)
    print(">>> await mw.aafter_agent(state, runtime)  (evict: worker.close)")
    state = {"_quickjs_snapshot_payload": None}  # thread mode w/o payload -> evict branch
    await mw.aafter_agent(state, object())  # runtime is unused by the hook
    print(">>> aafter_agent returned")

    hb.cancel()


asyncio.run(main())

Error Message and Stack Trace (if applicable)

Description

Summary

CodeInterpreterMiddleware maintains a pool of REPL "slots", each backed by a ThreadWorker (a dedicated worker thread + its own QuickJS context). On slot eviction (aevict) and slot close (_aclose_slot), the library calls ThreadWorker.close() directly on the asyncio event-loop thread. ThreadWorker.close() performs self._thread.join(timeout=5.0) plus a gc.collect() round-trip to the worker thread — a synchronous join that blocks the loop for up to ~5 s per eviction. Slot build has the same problem: _build_slot_locked calls worker.run_sync(...) on the loop thread, which blocks during worker init/GC.

Because CodeInterpreterMiddleware runs on every agent turn, these blocks recur frequently and can stall the loop for multiple seconds each time. While the loop is blocked, every asyncio.wait_for / asyncio.timeout bound on the same loop silently fails to fire.

Root-cause locations

quickjs_rs/threading.py:87-103 — ThreadWorker.close() is synchronous and blocking:

def close(self) -> None:
    """Stop the worker thread and join. Idempotent."""
    ...
    self._loop.call_soon_threadsafe(self._loop.stop)
    self._thread.join(timeout=5.0)   # ← blocks the calling (event-loop) thread
    self._loop = None
    self._thread = None

langchain_quickjs/_repl.py:1040-1045 — _aclose_slot (an async def reached from aevict) calls the blocking close() directly:

async def _aclose_slot(self, slot: _Slot) -> None:
    with contextlib.suppress(Exception):
        await slot.repl.aclose()
    with contextlib.suppress(Exception):
        await slot.worker.run_async(_aclose_runtime(slot.runtime))
    slot.worker.close()   # ← blocking, on the loop thread

langchain_quickjs/_repl.py:1014-1027 — _build_slot_locked runs blocking run_sync on the calling thread, and is reached from get():

def _build_slot_locked(self, thread_id: str) -> _Slot:
    ...
    worker = ThreadWorker(name=name)
    runtime = worker.run_sync(self._acreate_runtime())  # ← blocking init, on the loop thread
    repl = _ThreadREPL(worker, runtime, ...)
    return _Slot(worker=worker, runtime=runtime, repl=repl)

langchain_quickjs/middleware.py:379, 462 — both async hooks call the synchronous self._registry.get(thread_id), which calls _build_slot_locked on the loop thread on a cache miss:

# abefore_agent:379
repl = self._registry.get(thread_id)          # blocking on miss
# _prepare_for_call:462 (called from awrap_model_call:412)
repl = self._registry.get(thread_id)          # blocking on miss

Suggested fix

Offload every blocking ThreadWorker call to a worker thread via asyncio.to_thread so the event loop stays responsive:

  1. _aclose_slot — wrap the final slot.worker.close() in await asyncio.to_thread(slot.worker.close). (The preceding run_async already returns a future, so that part is fine.)
  2. Add an async aget(thread_id) to _Registry that builds the slot via asyncio.to_thread(worker.run_sync, …) / asyncio.to_thread(_ThreadREPL, …), with a double-checked lock (re-check _slots under the lock after the await; a race-loser schedules background cleanup of its duplicate slot rather than leaking it). The error path's cleanup worker.close() must also go through to_thread.
  3. middleware.abefore_agent / awrap_model_call — use await self._registry.aget(thread_id) instead of the synchronous get() so the first-build (cache-miss) path doesn't block the loop. (awrap_model_call can pre-warm the slot via aget before delegating to the original handler, so the subsequent sync _prepare_for_call is a cache hit.)

Happy to contribute a PR with these changes if there's interest.

Environment / System Info

  • langchain-quickjs 0.3.5
  • quickjs-rs 0.2.5
  • Python 3.12, asyncio loop (uvicorn --loop asyncio --workers 1)
  • Observed on MacOs and Linux

Activity

  1. added
    integration:quickjsIntegration with the QuickJS sandbox package.
    org:externalIssue or pull request created by someone outside the `langchain-ai` GitHub organization.
    on Aug 7, 2026
  2. uuzzrm commented on Aug 12, 2026

    @uuzzrm

    I traced the async path through the QuickJS registry and would like to work on this.

    The fix I have locally is intentionally scoped to the partner package:

    • add an async registry accessor that builds a cache-miss slot in asyncio.to_thread, then double-checks the registry and cleans up a duplicate slot if another caller won the race;
    • use that accessor from abefore_agent and the async model-call/PTC path;
    • keep async REPL tool installation and mode="call" reset off the caller event loop;
    • move the final ThreadWorker.close() in _aclose_slot to asyncio.to_thread;
    • add regression tests that hold worker initialization and shutdown long enough to prove the event loop still schedules callbacks.

    The synchronous API remains unchanged. The local QuickJS unit suite passes (218 tests), along with Ruff and ty. I have a branch ready at uuzzrm/quickjs/async-worker-lifecycle and can open/reopen the PR once this approach is confirmed. If this direction fits, please assign the issue to me; I am happy to adjust the implementation based on review.

  3. uuzzrm commented on Aug 14, 2026

    @uuzzrm

    I looked into this one and have a fix ready in PR #5441 (currently closed pending assignment, which I understand is the process here).

    The root cause: the async middleware path calls ThreadWorker.run_sync() and ThreadWorker.close() directly, and both do blocking work (building the worker on cache miss, joining threads on close) inside the event loop.

    The approach in the PR:

    • keep run_sync() for the synchronous API,
    • route async middleware through the async registry and REPL helpers,
    • offload cache-miss builds and close() with asyncio.to_thread so the loop keeps ticking,
    • keep run_async() on the worker loop,
    • and clean up duplicate slots on concurrent cache misses.

    Regression tests hold worker init and shutdown long enough to verify scheduled callbacks still run while those operations are in flight. 218 unit tests pass plus ruff and ty on the touched packages.

    Happy to adjust the design if the maintainers see a better fit.

  4. changed the title [-]langchain-quickjs: ThreadWorker.close() and ThreadWorker.run_sync() block the asyncio event loop[/-] [+]`ThreadWorker.close()` and `ThreadWorker.run_sync()` block the asyncio event loop[/+] on Aug 24, 2026
  5. removed
    type:bugAn unexpected problem or incorrect behavior.
    on Sep 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

integration:quickjsIntegration with the QuickJS sandbox package.org:externalIssue or pull request created by someone outside the `langchain-ai` GitHub organization.priority:backlogNot currently planned/prioritized work, often affecting a limited feature or set of users.topic:code-interpreterQuickJS and code interpreter middleware.type:bugAn unexpected problem or incorrect behavior.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions