Skip to content

Add optional fallback runtimes for agents#5282

Draft
sangemaru wants to merge 9 commits into
multica-ai:mainfrom
sangemaru:codex/optional-fallback-runtimes
Draft

Add optional fallback runtimes for agents#5282
sangemaru wants to merge 9 commits into
multica-ai:mainfrom
sangemaru:codex/optional-fallback-runtimes

Conversation

@sangemaru

@sangemaru sangemaru commented Jul 11, 2026

Copy link
Copy Markdown

Closes #1398

What changed

  • adds an ordered, optional fallback-runtime chain to agent create/update APIs, CLI commands, and the agent UI
  • applies the same workspace visibility and private-runtime ownership authorization to every fallback as to the primary runtime
  • routes fresh issue, mention, squad, quick-create, chat, and Autopilot tasks away from an offline or actively cooled primary
  • recognizes only provider-shaped quota, credit, usage-cap, and terminal capacity/rate-limit failures; ordinary coding, configuration, or prose mentioning quota or credits remains non-retryable
  • walks a bounded fallback chain with ancestry-wide cycle prevention, per-runtime cooldowns, race-safe recovery, and idempotent history/Inbox notifications
  • preserves issue, chat input ordering, squad role, Autopilot run lineage, overlays, and task attribution across retry children
  • starts a fresh provider session when crossing runtimes while preserving the worktree only when the destination provider runtime is registered on the same daemon and can actually access that host-local path
  • hands cross-runtime context over without prompt injection: the daemon writes a bounded provider-neutral transcript to read-only .multica/fallback-context/<source-task>/transcript.jsonl, and the opening brief contains only its relative file pointer and source task ID
  • keeps same-runtime native session resume unchanged
  • adds foreign-key-free fallback state tables with application-layer teardown cleanup and one CREATE INDEX CONCURRENTLY statement per index migration (202 through 208)

Issue #1398 acceptance criteria

The retry-child path intentionally includes canonical quota/capacity failure reasons (agent_error.provider_quota_limit and agent_error.provider_capacity_or_rate_limit). It does not retry generic agent_error failures.

For the concern in issue comment 4976910822:

  • quota/credit/usage-cap and terminal capacity/rate-limit failures can enter the configured cross-provider chain;
  • the classifier uses narrow provider signatures and has negative tests for ordinary quota/credit prose;
  • attempts are finite, cycle-safe, and cooldown-aware;
  • runtime crossing always starts a fresh provider session;
  • the transcript is a read-only file handoff and is never inserted into the prompt;
  • an in-flight worktree is offered only to a provider runtime on the same registered daemon; a cross-daemon regression test proves a host-local path is not handed to another machine;
  • issue, chat, squad, quick-create, mention, and Autopilot behavior is covered;
  • execution history and Inbox events identify the canonical failure and source/destination runtimes;
  • an exhausted primary is temporarily suppressed for new tasks and becomes eligible again after cooldown or a qualifying later success.

Validation

The branch is based directly on current main commit 002ea0d87949d112d96586bd8b42c779142cf77d.

  • independent bounded read-only review with gpt-5.6-terra; all four material findings resolved
  • fresh pgvector/pgvector:pg17 database migration from 001 through 208
  • complete migration down/up round trip
  • go build ./...
  • go test -race ./...
  • frontend typecheck and lint across all non-docs, non-mobile packages
  • 4,104 frontend tests: core 987, views 2,680, web 108, desktop 329
  • production web and desktop builds with Next's official Google Fonts transport mock
  • git diff --check

The real-workflow proof used an isolated Multica workspace, real agents, two different provider runtimes, and real issues. A controlled primary quota failure produced a 15-minute cooldown, queued a bounded child on the fallback provider, preserved the same worktree with a fresh provider session, materialized a mode-0444 JSONL transcript, and completed through the fallback. Separate real tasks proved that an active cooldown bypasses the primary and that the primary becomes eligible after expiry. Full task IDs, runtime IDs, database rows, Inbox/history evidence, and commands are recorded in docs/plans/fallback-runtime-live-smoke-2026-07-11.md.

Remaining gate

This PR remains draft until the newly triggered upstream CI and Mobile Verify checks pass. The local unmocked web build was unable to complete the large Google Fonts fetch on this Mac; no source workaround was added, so upstream frontend CI remains the authoritative unmocked transport check.

@vercel

vercel Bot commented Jul 11, 2026

Copy link
Copy Markdown

@sangemaru is attempting to deploy a commit to the IndexLabs Team on Vercel.

A member of the Team first needs to authorize it.

@sangemaru
sangemaru force-pushed the codex/optional-fallback-runtimes branch from 1acbf01 to 13c6f1d Compare July 18, 2026 22:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Add optional Fallback Runtime

1 participant