Skip to content

feat: add DB-backed OAuth token storage for Google toolkits#7376

Open
Mustafa-Esoofally wants to merge 24 commits intomainfrom
feat/google-auth-db-storage
Open

feat: add DB-backed OAuth token storage for Google toolkits#7376
Mustafa-Esoofally wants to merge 24 commits intomainfrom
feat/google-auth-db-storage

Conversation

@Mustafa-Esoofally
Copy link
Copy Markdown
Contributor

@Mustafa-Esoofally Mustafa-Esoofally commented Apr 6, 2026

Summary

Persist Google OAuth tokens in the agent's database. Tokens survive server restarts and load automatically on subsequent runs.

Usage

Simple mode — save token to DB

from agno.agent import Agent
from agno.db.sqlite.sqlite import SqliteDb
from agno.tools.google.gmail import GmailTools

agent = Agent(
    tools=[GmailTools(store_token_in_db=True)],
    db=SqliteDb(db_file="tmp/agent.db"),
)
agent.run("check my email")

First run opens a browser for OAuth. Token saves to DB only (not token.json). Next run loads from DB — no browser needed.

GoogleAuth mode — multi-toolkit with consolidated scopes

from agno.tools.google.auth import GoogleAuth
from agno.tools.google.calendar import GoogleCalendarTools

google_auth = GoogleAuth()
agent = Agent(
    tools=[
        google_auth,
        GmailTools(google_auth=google_auth),
        GoogleCalendarTools(google_auth=google_auth),
    ],
    db=SqliteDb(db_file="tmp/agent.db"),
)

GoogleAuth consolidates scopes across toolkits. When no token exists, the agent returns an OAuth URL for the user to visit (designed for Slack/WhatsApp interfaces).

Default — unchanged

agent = Agent(tools=[GmailTools()])  # file-based token.json, same as before

Flow

agent.run("check email", user_id="alice")
  │
  ├─ get_tools() auto-wires: toolkit._db = agent.db
  │
  ├─ model calls get_latest_emails()
  │   └─ @google_authenticate decorator
  │       └─ _auth(user_id="alice")
  │           │
  │           ├─ load_token() → DB lookup by (provider, user_id, service)
  │           │   ├─ found + valid → use cached creds ✅
  │           │   ├─ found + expired → refresh via refresh_token → re-persist ✅
  │           │   └─ not found → continue below
  │           │
  │           ├─ GoogleAuth mode: raise PermissionError → agent returns OAuth URL
  │           └─ Simple mode: file-based OAuth → browser → save to DB
  │
  └─ Gmail API call with valid credentials

Token refresh

Users authenticate once. Expired tokens auto-refresh:

  1. load_token() → DB lookup → creds.expired = True
  2. creds.refresh(Request()) → Google returns new access_token (~200ms)
  3. save_token() → updated token persisted to DB
  4. No user interaction needed

DB schema

CREATE TABLE agno_auth_tokens (
    id         VARCHAR PRIMARY KEY,  -- deterministic: "provider:user_id:service"
    provider   VARCHAR NOT NULL,     -- "google"
    user_id    VARCHAR,              -- nullable, from agent.run(user_id=...)
    service    VARCHAR NOT NULL,     -- "google" (consolidated across all Google APIs)
    token_data JSON NOT NULL,        -- serialized Google Credentials
    granted_scopes JSON,
    created_at BIGINT NOT NULL,
    updated_at BIGINT
);

Schema follows the codebase pattern: id as PK (like sessions, memories), user_id nullable (like all other tables), BigInteger timestamps.

Files changed

Layer Files
DB schema + CRUD base.py, postgres/schemas.py, postgres.py, async_postgres.py, sqlite/schemas.py, sqlite.py, async_sqlite.py
Migration manager migrations/manager.py
Auth helpers tools/google/auth.pyload_token, save_token, get_token_db, GoogleAuth toolkit
Google toolkits gmail.py, calendar.py, drive.py, sheets.py, slides.pystore_token_in_db flag, _db attribute
Auto-wiring agent/_tools.py, team/_tools.py — binds agent.db to toolkit _db
Cookbooks gmail_with_db.py, google_workspace_agent.py, google_auth_db_storage.py, + DB examples in existing cookbooks

Type of change

  • New feature

Checklist

  • Ran format/validation scripts
  • Both sync and async DB implementations
  • Backward compatible — file-based OAuth unaffected
  • E2E tested with real Gmail API
  • Cookbooks included

Add per-user OAuth token persistence using the agent's existing database.
When GoogleAuth is paired with a DB-backed agent, tokens are stored in
agno_auth_tokens and automatically loaded/refreshed on subsequent runs.
File-based token.json remains the default for cookbooks with no DB.

Changes:
- BaseDb: add get_auth_token/upsert_auth_token/delete_auth_token stubs
- Postgres + SQLite: implement agno_auth_tokens table with CRUD
- GoogleAuth: add load_token, store_token, handle_oauth_callback,
  get_oauth_router for OAuth callback endpoint
- google_authenticate decorator: add RLock for thread safety, extract
  user_id from RunContext, inject run_context into wrapper signature
- All Google toolkits: accept google_auth param, delegate auth to
  GoogleAuth DB store before falling back to file-based flow
- Auto-wire agent.db to toolkit._db during get_tools()
Only wire toolkit._db when the backend is a sync BaseDb that actually
overrides get_auth_token — prevents coroutine-object errors with async
DBs and avoids re-auth loops on backends without token support. When
_db is not wired, toolkits fall through to file-based OAuth (the
pre-existing default). Also fix PostgresDb.from_dict() to round-trip
auth_tokens_table.
- Extract google_auth_or_raise() to replace 4-line PermissionError
  pattern duplicated across all 5 Google toolkits
- Extract wire_db_to_toolkits() to replace auto-wiring block
  duplicated in agent/_tools.py (sync + async) and team/_tools.py
- Restore accidentally removed team.skills.get_tools() registration
- Fix XSS: validate + escape services in callback success HTML
- Handle Google error/error_description params in OAuth callback
- store_token() returns bool; callback fails if DB persistence fails
@Mustafa-Esoofally Mustafa-Esoofally force-pushed the feat/google-auth-db-storage branch from 7b8fc3e to 999a1a4 Compare April 6, 2026 19:19
…ples

- google_workspace_agent.py: Gmail + Calendar + Drive with consolidated
  scopes via GoogleAuth coordinator, file-based token.json
- google_auth_db_storage.py: DB-backed token persistence with SQLite,
  demonstrates store/load roundtrip and multi-user isolation
@Mustafa-Esoofally Mustafa-Esoofally force-pushed the feat/google-auth-db-storage branch from 03c23a5 to 8d66ca0 Compare April 6, 2026 19:31
Single-toolkit DB storage without GoogleAuth:
  GmailTools(store_token_in_db=True) + Agent(db=SqliteDb(...))
  -> file-based OAuth opens browser as usual
  -> token saved to BOTH token.json AND DB
  -> subsequent runs load from DB first (no browser)

GoogleAuth remains for multi-toolkit scope consolidation and
interface OAuth URL flow (Slack/WhatsApp).

Also fixes GmailTools port=None crash (changed default to 0).
Add commented DB variant to gmail_tools, calendar_event_creator,
drive_tools, slide_tools, and googlesheets_tools showing
store_token_in_db=True usage. Remove test_db_oauth_roundtrip.py.
Copy link
Copy Markdown
Contributor Author

@Mustafa-Esoofally Mustafa-Esoofally left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Design Review — DB-Backed OAuth Token Storage

Inline comments explain the architecture decisions for each layer. Summary of the design:

Architecture Layers

┌─────────────────────────────────────────────────────────┐
│ Agent._tools.py — auto-wiring (DB → toolkit._db)        │
│   Guards: sync BaseDb only, must override get_auth_token │
├─────────────────────────────────────────────────────────┤
│ auth.py — decorator + helpers                            │
│   @google_authenticate: RLock, run_context, user_id     │
│   google_auth_or_raise: two-mode (GoogleAuth vs simple) │
│   google_auth_save_to_store: persists after file auth   │
├─────────────────────────────────────────────────────────┤
│ GoogleAuth toolkit — coordinator + OAuth callback        │
│   load_token/store_token: DB CRUD + auto-refresh        │
│   handle_oauth_callback: code→creds→store               │
│   get_oauth_router: FastAPI endpoint                     │
├─────────────────────────────────────────────────────────┤
│ DB layer — BaseDb stubs + Postgres/SQLite implementations│
│   Composite PK: (provider, user_id, service)            │
│   Upsert with ON CONFLICT DO UPDATE                     │
│   Token consolidation: service="google" for all APIs    │
└─────────────────────────────────────────────────────────┘

Two Auth Modes

GoogleAuth mode (interfaces — Slack, WhatsApp):
GoogleAuth() + google_auth= on toolkits + agent.db
→ No token? PermissionError → agent returns OAuth URL → callback stores token

Simple mode (single toolkit):
GmailTools(store_token_in_db=True) + agent.db
→ No token? Fall through to browser OAuth → saves to BOTH file + DB

Token Refresh (no re-auth needed)

  1. First auth: user consents → access_token + refresh_token → stored in DB
  2. Token valid (~1hr): load_token() → return immediately
  3. Token expired: load_token() → refresh(Request()) → re-persist → return
  4. Refresh tokens don't expire unless revoked

Known Limitations

  • Async DB: Auto-wiring skips AsyncBaseDb — sync PostgresDb works in all contexts (async tools use asyncio.to_thread)
  • State param: Base64-encoded, not HMAC-signed (follow-up for CSRF protection)
  • Shared instance: Creds cached on shared toolkit object — follow-up PR adds per-request clone()

When store_token_in_db=True or google_auth is set, token is saved to DB
only. File-based token.json is only written when no DB path is active.
google_auth_save_to_store returns bool so callers know which path ran.

Also fix async_sqlite get_auth_token missing begin() transaction wrapper.
The decorator extracts user_id from run_context (injected by the
framework via signature inspection) and passes it to _auth(). This
ensures each user_id gets its own token row in the DB.

Adds gmail_multiuser_db.py cookbook demonstrating per-user isolation.
…nderscore prefix

- get_token_db: resolves DB from GoogleAuth or store_token_in_db flag
- load_token: load from DB, refresh if expired, set toolkit.creds
- save_token: persist to DB

PermissionError for GoogleAuth mode is now inline in each toolkit's
_auth() — visible, not hidden behind indirection.

Removed: google_auth_from_store, google_auth_or_raise,
google_auth_save_to_store, _load_token_from_db, _save_token_to_db
…rom toolkits

Decorator is identical to main — no inspect, no signature manipulation.
user_id for DB lookups comes from GoogleAuth.user_id (set at construction)
or empty string for single-user cookbooks. Per-request user_id via
run_context is deferred to the interface/isolation follow-up PR.
user_id flows via the framework's existing DI:
  agent.run(user_id="x") → RunContext → run_context injected into
  decorator via __signature__ override → passed to _auth(user_id=)

Removed: _user_id monkey-patching on toolkits, _resolve_user_id helper.
Matches codebase pattern — all other tables use a single string id
as PK with user_id as a nullable indexed column.

id is generated deterministically from (provider, user_id, service)
so upserts work via ON CONFLICT (id). user_id=None for anonymous.
if agent.db is not None and resolved_tools:
from agno.db.base import BaseDb

if isinstance(agent.db, BaseDb) and type(agent.db).get_auth_token is not BaseDb.get_auth_token:
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Auto-wiring: Runs during tool resolution (before any tool call). Sets toolkit._db = agent.db so the auth helpers can read/write tokens.

Guards:

  • isinstance(agent.db, BaseDb) — skips AsyncBaseDb (auth helpers are sync-only)
  • type(db).get_auth_token is not BaseDb.get_auth_token — skips DBs that don't implement token storage
  • tool._db is None — won't overwrite explicit db= on GoogleAuth

return json.dumps({"error": f"{service_name.title()} service initialization failed: {e}"})
return func(self, *args, **kwargs)

wrapper.__signature__ = exposed_sig
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Three helpers:

  • get_token_db — resolves DB from GoogleAuth._db or toolkit._db (store_token_in_db mode)
  • load_token — DB read + auto-refresh expired tokens + re-persist
  • save_token — DB write with deterministic id

user_id flows from run_context (injected by framework). None when no user_id set on agent.

"run_status": {"type": String, "nullable": True, "index": True},
}

AUTH_TOKEN_TABLE_SCHEMA = {
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Schema matches codebase pattern: id as PK (like sessions, memories, learnings), user_id nullable (like all other tables). id is deterministic from provider:user_id:service so upserts use ON CONFLICT (id).


user_id = state_data.get("user_id", "")
services = state_data.get("services", [])

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Callback security:

  • Google error params surfaced (not hidden as "missing code")
  • store_token returns bool — callback fails if DB write fails
  • Services validated against registered allow-list before HTML rendering
  • All output HTML-escaped

Known limitation: state is base64 JSON, not HMAC-signed (follow-up for CSRF).


def _auth(self) -> None:
"""Authenticate with Gmail API using service account (priority) or OAuth flow."""
def _auth(self, user_id=None) -> None:
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Auth priority in each toolkit's _auth():

  1. self.creds already valid → return
  2. Service account → never stored in DB
  3. load_token() → tries DB (GoogleAuth mode or store_token_in_db mode)
  4. GoogleAuth + DB but no token → PermissionError (agent returns OAuth URL)
  5. File-based OAuth → browser opens
  6. After auth: save_token() → DB only. File written only if no DB path.

# Expose original typed params + run_context in the signature so the framework:
# (1) builds the correct LLM tool schema from the original params
# (2) injects run_context (which carries user_id) at call time
# run_context is stripped from the LLM schema at function.py:660-671
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why signature override: @wraps sets wrapped which makes inspect.signature follow to the original function. Without the override, the framework would not see run_context in the signature and would not inject it. The override preserves the original typed params (for LLM schema) and adds run_context (for framework DI). run_context is stripped from the LLM schema at function.py:660-671.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant