Skip to content

feat: per-request toolkit isolation for concurrent multi-user requests#7377

Open
Mustafa-Esoofally wants to merge 1 commit intofeat/google-auth-db-storagefrom
feat/toolkit-isolation-v2
Open

feat: per-request toolkit isolation for concurrent multi-user requests#7377
Mustafa-Esoofally wants to merge 1 commit intofeat/google-auth-db-storagefrom
feat/toolkit-isolation-v2

Conversation

@Mustafa-Esoofally
Copy link
Copy Markdown
Contributor

Summary

Add clone-based isolation so each interface request (Slack, Telegram, WhatsApp) gets fresh toolkit state while sharing heavy resources by reference.

Depends on: #7376

Problem

When multiple users message a Slack bot concurrently, they share the same agent instance. Google toolkits hold mutable state (creds, service, _label_cache) that can leak between users — User A's Gmail token could be used for User B's request.

Solution

Each request clones the agent via entity.deep_copy() before calling arun(). Per-user state is reset; shared resources (DB, GoogleAuth, MCP connections) are kept by reference.

Slack message arrives
  │
  ▼
entity.deep_copy()
  ├─ GoogleAuth: SHARED by reference (DB coordinator, like MCP tools)
  ├─ GmailTools: CLONED via clone(), resets creds/service/_label_cache
  ├─ MCP tools: SHARED by reference (server connections)
  └─ DB, model: SHARED by reference (connection pools)
  │
  ▼
cloned_agent.arun(message, user_id=...)
  └─ Fresh creds loaded from DB for this user

Init-time DB wiring

_initialize_agents() wires toolkit._db = agent.db at app startup (not runtime). This ensures the OAuth callback closure has DB access before any request arrives — no app.state.db needed.

Files changed (10 files, +382 -21)

Layer Files
Clone infra toolkit.py (clone + _clone_reset_attrs), function.py (clone_for)
deep_copy agent/_utils.py, team/_utils.py (share GoogleAuth by ref)
Routers slack/router.py, telegram/router.py, whatsapp/router.py
App wiring os/app.py (init-time _db binding)
Cookbook slack/gmail_agent.py
Tests test_toolkit_isolation.py

Type of change

  • New feature

Checklist

  • Code complies with style guidelines
  • Ran format/validation scripts
  • Self-review completed
  • Cookbook example included
  • Tests added (test_toolkit_isolation.py)
  • E2E tested: Slack → OAuth URL → Google consent → DB token → retry → emails displayed

Duplicate and AI-Generated PR Check

  • No existing PR addresses this
  • This PR was developed with AI assistance (Claude Code)

Additional Notes

  • E2E verified: Full Slack OAuth flow tested with ngrok + PgVector + Gmail API
  • Pattern: GoogleAuth follows the same shared-by-reference lifecycle as MCP tools
  • All interfaces covered: Slack, Telegram, WhatsApp routers all call deep_copy()

@Mustafa-Esoofally Mustafa-Esoofally requested a review from a team as a code owner April 6, 2026 17:56
@Mustafa-Esoofally Mustafa-Esoofally force-pushed the feat/toolkit-isolation-v2 branch from 0f8d56e to 1fe64da Compare April 6, 2026 18:30
Add clone-based isolation so each Slack/Telegram/WhatsApp request gets
fresh toolkit state (creds, service, caches) while sharing heavy
resources (DB, GoogleAuth, MCP connections) by reference.

Changes:
- Toolkit: add clone() with _clone_reset_attrs for per-user state reset
- Function: add clone_for() to rebind entrypoints to cloned toolkit
- deep_copy_field: share GoogleAuth by reference (like MCP tools),
  fall back to clone() when deepcopy fails (Google httplib2 clients)
- Interface routers: call entity.deep_copy() before arun() so each
  request runs on an isolated agent copy
- AgentOS: wire toolkit._db at init time (_initialize_agents) so the
  OAuth callback closure has DB access before any request arrives
- Cookbook: Slack Gmail agent with DB-backed OAuth flow

Depends on: feat/google-auth-db-storage
@Mustafa-Esoofally Mustafa-Esoofally force-pushed the feat/toolkit-isolation-v2 branch from 1fe64da to 5eaf013 Compare April 6, 2026 22:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant