Skip to content

Add Tinfoil private inference provider#875

Merged
gabrielste1n merged 3 commits into
OpenWhispr:mainfrom
zachdotai:feat/tinfoil-private-inference
Jul 6, 2026
Merged

Add Tinfoil private inference provider#875
gabrielste1n merged 3 commits into
OpenWhispr:mainfrom
zachdotai:feat/tinfoil-private-inference

Conversation

@zachdotai

Copy link
Copy Markdown
Contributor

Summary

Adds Tinfoil as a first-class private inference provider for OpenWhispr LLM workflows.

This integration uses the official Tinfoil SDK instead of the generic OpenAI-compatible fetch path so requests go through Tinfoil’s verified enclave transport. Tinfoil is now available for cleanup, note formatting, dictation agent, and chat agent flows, including AI SDK streaming/tool usage.

Changes

  • Add tinfoil SDK dependency.
  • Add a lazy-loaded Tinfoil client wrapper and non-streaming inference provider.
  • Register Tinfoil in the provider registry and model catalog.
  • Add secure Tinfoil API key storage through the existing encrypted secret/IPC/settings flow.
  • Add Tinfoil to reasoning provider UI and Prompt Studio provider metadata.
  • Route chat agent streaming/tools through the Tinfoil AI SDK provider.
  • Prevent raw manual streaming from using Tinfoil, avoiding an unverified transport path.
  • Document TINFOIL_API_KEY and Tinfoil network allowlist hosts.

Test Plan

  • npm run typecheck
  • npm run i18n:check
  • npm run lint
  • npm run build:renderer
  • git diff --check
  • Verified Tinfoil appears in the reasoning provider picker with API key storage and model selection wired through the settings store.
  • Verified Tinfoil cleanup/chat paths use the official SDK clients instead of the raw OpenAI-compatible fetch path, so enclave verification is preserved.
  • Verified non-Tinfoil provider paths still compile after making the renderer AI model factory async.

zachdotai and others added 3 commits May 31, 2026 01:08
# Conflicts:
#	README.md
#	docs/network-allowlist.md
#	package-lock.json
#	package.json
#	preload.js
#	src/helpers/environment.js
#	src/helpers/ipcHandlers.js
#	src/hooks/useSettings.ts
#	src/services/ReasoningService.ts
#	src/stores/settingsStore.ts
#	src/types/electron.ts
- Update model catalog: glm-5-1 is now glm-5-2 (384K context) per
  Tinfoil's current chat model list
- Add descriptionKey i18n wiring for all six models across the ten
  locales, matching the groq registry pattern
- Add supportsThinking to the reasoning-capable models so the
  disable-thinking toggle and suppression work like they do for groq
- Cap each request attempt at 30s and disable SDK-internal retries so
  withRetry stays the single retry layer, matching sibling providers
- Retry the tinfoil module import after a failed load instead of
  caching the rejection
- Drop the redundant per-call verification (the client attests before
  every request) and an unneeded LanguageModel cast
- Document the dynamically assigned enclave host in the network
  allowlist and drop hosts the app never contacts
- Point console links at tinfoil.sh/inference with referral params
@gabrielste1n gabrielste1n self-requested a review July 6, 2026 14:38

@gabrielste1n gabrielste1n left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

absolutely legendary thank you sir @zachdotai

@gabrielste1n gabrielste1n merged commit 880f4f6 into OpenWhispr:main Jul 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants