feat: add DB-backed OAuth token storage for Google toolkits#7376
feat: add DB-backed OAuth token storage for Google toolkits#7376Mustafa-Esoofally wants to merge 24 commits intomainfrom
Conversation
Add per-user OAuth token persistence using the agent's existing database. When GoogleAuth is paired with a DB-backed agent, tokens are stored in agno_auth_tokens and automatically loaded/refreshed on subsequent runs. File-based token.json remains the default for cookbooks with no DB. Changes: - BaseDb: add get_auth_token/upsert_auth_token/delete_auth_token stubs - Postgres + SQLite: implement agno_auth_tokens table with CRUD - GoogleAuth: add load_token, store_token, handle_oauth_callback, get_oauth_router for OAuth callback endpoint - google_authenticate decorator: add RLock for thread safety, extract user_id from RunContext, inject run_context into wrapper signature - All Google toolkits: accept google_auth param, delegate auth to GoogleAuth DB store before falling back to file-based flow - Auto-wire agent.db to toolkit._db during get_tools()
Only wire toolkit._db when the backend is a sync BaseDb that actually overrides get_auth_token — prevents coroutine-object errors with async DBs and avoids re-auth loops on backends without token support. When _db is not wired, toolkits fall through to file-based OAuth (the pre-existing default). Also fix PostgresDb.from_dict() to round-trip auth_tokens_table.
- Extract google_auth_or_raise() to replace 4-line PermissionError pattern duplicated across all 5 Google toolkits - Extract wire_db_to_toolkits() to replace auto-wiring block duplicated in agent/_tools.py (sync + async) and team/_tools.py - Restore accidentally removed team.skills.get_tools() registration - Fix XSS: validate + escape services in callback success HTML - Handle Google error/error_description params in OAuth callback - store_token() returns bool; callback fails if DB persistence fails
7b8fc3e to
999a1a4
Compare
…ples - google_workspace_agent.py: Gmail + Calendar + Drive with consolidated scopes via GoogleAuth coordinator, file-based token.json - google_auth_db_storage.py: DB-backed token persistence with SQLite, demonstrates store/load roundtrip and multi-user isolation
03c23a5 to
8d66ca0
Compare
Single-toolkit DB storage without GoogleAuth: GmailTools(store_token_in_db=True) + Agent(db=SqliteDb(...)) -> file-based OAuth opens browser as usual -> token saved to BOTH token.json AND DB -> subsequent runs load from DB first (no browser) GoogleAuth remains for multi-toolkit scope consolidation and interface OAuth URL flow (Slack/WhatsApp). Also fixes GmailTools port=None crash (changed default to 0).
Add commented DB variant to gmail_tools, calendar_event_creator, drive_tools, slide_tools, and googlesheets_tools showing store_token_in_db=True usage. Remove test_db_oauth_roundtrip.py.
Mustafa-Esoofally
left a comment
There was a problem hiding this comment.
Design Review — DB-Backed OAuth Token Storage
Inline comments explain the architecture decisions for each layer. Summary of the design:
Architecture Layers
┌─────────────────────────────────────────────────────────┐
│ Agent._tools.py — auto-wiring (DB → toolkit._db) │
│ Guards: sync BaseDb only, must override get_auth_token │
├─────────────────────────────────────────────────────────┤
│ auth.py — decorator + helpers │
│ @google_authenticate: RLock, run_context, user_id │
│ google_auth_or_raise: two-mode (GoogleAuth vs simple) │
│ google_auth_save_to_store: persists after file auth │
├─────────────────────────────────────────────────────────┤
│ GoogleAuth toolkit — coordinator + OAuth callback │
│ load_token/store_token: DB CRUD + auto-refresh │
│ handle_oauth_callback: code→creds→store │
│ get_oauth_router: FastAPI endpoint │
├─────────────────────────────────────────────────────────┤
│ DB layer — BaseDb stubs + Postgres/SQLite implementations│
│ Composite PK: (provider, user_id, service) │
│ Upsert with ON CONFLICT DO UPDATE │
│ Token consolidation: service="google" for all APIs │
└─────────────────────────────────────────────────────────┘
Two Auth Modes
GoogleAuth mode (interfaces — Slack, WhatsApp):
GoogleAuth() + google_auth= on toolkits + agent.db
→ No token? PermissionError → agent returns OAuth URL → callback stores token
Simple mode (single toolkit):
GmailTools(store_token_in_db=True) + agent.db
→ No token? Fall through to browser OAuth → saves to BOTH file + DB
Token Refresh (no re-auth needed)
- First auth: user consents → access_token + refresh_token → stored in DB
- Token valid (~1hr): load_token() → return immediately
- Token expired: load_token() → refresh(Request()) → re-persist → return
- Refresh tokens don't expire unless revoked
Known Limitations
- Async DB: Auto-wiring skips AsyncBaseDb — sync PostgresDb works in all contexts (async tools use asyncio.to_thread)
- State param: Base64-encoded, not HMAC-signed (follow-up for CSRF protection)
- Shared instance: Creds cached on shared toolkit object — follow-up PR adds per-request clone()
When store_token_in_db=True or google_auth is set, token is saved to DB only. File-based token.json is only written when no DB path is active. google_auth_save_to_store returns bool so callers know which path ran. Also fix async_sqlite get_auth_token missing begin() transaction wrapper.
The decorator extracts user_id from run_context (injected by the framework via signature inspection) and passes it to _auth(). This ensures each user_id gets its own token row in the DB. Adds gmail_multiuser_db.py cookbook demonstrating per-user isolation.
…nderscore prefix - get_token_db: resolves DB from GoogleAuth or store_token_in_db flag - load_token: load from DB, refresh if expired, set toolkit.creds - save_token: persist to DB PermissionError for GoogleAuth mode is now inline in each toolkit's _auth() — visible, not hidden behind indirection. Removed: google_auth_from_store, google_auth_or_raise, google_auth_save_to_store, _load_token_from_db, _save_token_to_db
…rom toolkits Decorator is identical to main — no inspect, no signature manipulation. user_id for DB lookups comes from GoogleAuth.user_id (set at construction) or empty string for single-user cookbooks. Per-request user_id via run_context is deferred to the interface/isolation follow-up PR.
user_id flows via the framework's existing DI: agent.run(user_id="x") → RunContext → run_context injected into decorator via __signature__ override → passed to _auth(user_id=) Removed: _user_id monkey-patching on toolkits, _resolve_user_id helper.
Matches codebase pattern — all other tables use a single string id as PK with user_id as a nullable indexed column. id is generated deterministically from (provider, user_id, service) so upserts work via ON CONFLICT (id). user_id=None for anonymous.
| if agent.db is not None and resolved_tools: | ||
| from agno.db.base import BaseDb | ||
|
|
||
| if isinstance(agent.db, BaseDb) and type(agent.db).get_auth_token is not BaseDb.get_auth_token: |
There was a problem hiding this comment.
Auto-wiring: Runs during tool resolution (before any tool call). Sets toolkit._db = agent.db so the auth helpers can read/write tokens.
Guards:
isinstance(agent.db, BaseDb)— skips AsyncBaseDb (auth helpers are sync-only)type(db).get_auth_token is not BaseDb.get_auth_token— skips DBs that don't implement token storagetool._db is None— won't overwrite explicitdb=on GoogleAuth
libs/agno/agno/tools/google/auth.py
Outdated
| return json.dumps({"error": f"{service_name.title()} service initialization failed: {e}"}) | ||
| return func(self, *args, **kwargs) | ||
|
|
||
| wrapper.__signature__ = exposed_sig |
There was a problem hiding this comment.
Three helpers:
get_token_db— resolves DB from GoogleAuth._db or toolkit._db (store_token_in_db mode)load_token— DB read + auto-refresh expired tokens + re-persistsave_token— DB write with deterministic id
user_id flows from run_context (injected by framework). None when no user_id set on agent.
| "run_status": {"type": String, "nullable": True, "index": True}, | ||
| } | ||
|
|
||
| AUTH_TOKEN_TABLE_SCHEMA = { |
There was a problem hiding this comment.
Schema matches codebase pattern: id as PK (like sessions, memories, learnings), user_id nullable (like all other tables). id is deterministic from provider:user_id:service so upserts use ON CONFLICT (id).
|
|
||
| user_id = state_data.get("user_id", "") | ||
| services = state_data.get("services", []) | ||
|
|
There was a problem hiding this comment.
Callback security:
- Google error params surfaced (not hidden as "missing code")
store_tokenreturns bool — callback fails if DB write fails- Services validated against registered allow-list before HTML rendering
- All output HTML-escaped
Known limitation: state is base64 JSON, not HMAC-signed (follow-up for CSRF).
|
|
||
| def _auth(self) -> None: | ||
| """Authenticate with Gmail API using service account (priority) or OAuth flow.""" | ||
| def _auth(self, user_id=None) -> None: |
There was a problem hiding this comment.
Auth priority in each toolkit's _auth():
self.credsalready valid → return- Service account → never stored in DB
load_token()→ tries DB (GoogleAuth mode or store_token_in_db mode)- GoogleAuth + DB but no token → PermissionError (agent returns OAuth URL)
- File-based OAuth → browser opens
- After auth:
save_token()→ DB only. File written only if no DB path.
| # Expose original typed params + run_context in the signature so the framework: | ||
| # (1) builds the correct LLM tool schema from the original params | ||
| # (2) injects run_context (which carries user_id) at call time | ||
| # run_context is stripped from the LLM schema at function.py:660-671 |
There was a problem hiding this comment.
Why signature override: @wraps sets wrapped which makes inspect.signature follow to the original function. Without the override, the framework would not see run_context in the signature and would not inject it. The override preserves the original typed params (for LLM schema) and adds run_context (for framework DI). run_context is stripped from the LLM schema at function.py:660-671.
Summary
Persist Google OAuth tokens in the agent's database. Tokens survive server restarts and load automatically on subsequent runs.
Usage
Simple mode — save token to DB
First run opens a browser for OAuth. Token saves to DB only (not token.json). Next run loads from DB — no browser needed.
GoogleAuth mode — multi-toolkit with consolidated scopes
GoogleAuth consolidates scopes across toolkits. When no token exists, the agent returns an OAuth URL for the user to visit (designed for Slack/WhatsApp interfaces).
Default — unchanged
Flow
Token refresh
Users authenticate once. Expired tokens auto-refresh:
load_token()→ DB lookup →creds.expired = Truecreds.refresh(Request())→ Google returns new access_token (~200ms)save_token()→ updated token persisted to DBDB schema
Schema follows the codebase pattern:
idas PK (like sessions, memories),user_idnullable (like all other tables),BigIntegertimestamps.Files changed
base.py,postgres/schemas.py,postgres.py,async_postgres.py,sqlite/schemas.py,sqlite.py,async_sqlite.pymigrations/manager.pytools/google/auth.py—load_token,save_token,get_token_db, GoogleAuth toolkitgmail.py,calendar.py,drive.py,sheets.py,slides.py—store_token_in_dbflag,_dbattributeagent/_tools.py,team/_tools.py— bindsagent.dbto toolkit_dbgmail_with_db.py,google_workspace_agent.py,google_auth_db_storage.py, + DB examples in existing cookbooksType of change
Checklist