Skip to content

Commit 57a3af0

Browse files
infiniVinfiniV
authored
Merge pull request #33 from infiniV/refactor/architecture-deepening
refactor(models): consolidate model catalog and deepen download flow
2 parents 0446d3d + c9d897c commit 57a3af0

51 files changed

Lines changed: 3876 additions & 2241 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/ci.yml‎

Lines changed: 132 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,132 @@
1+
name: CI
2+
3+
on:
4+
pull_request:
5+
push:
6+
branches: [main]
7+
8+
concurrency:
9+
group: ci-${{ github.ref }}
10+
cancel-in-progress: true
11+
12+
jobs:
13+
test:
14+
runs-on: ubuntu-22.04
15+
timeout-minutes: 20
16+
env:
17+
# Qt without a display server (UiEventBridge tests create a QCoreApplication)
18+
QT_QPA_PLATFORM: offscreen
19+
steps:
20+
- uses: actions/checkout@v4
21+
22+
- name: Install system dependencies
23+
run: |
24+
sudo apt-get update
25+
sudo apt-get install -y \
26+
libportaudio2 libasound2 \
27+
libgl1-mesa-dev libegl1-mesa-dev libxkbcommon0 libfontconfig1 \
28+
libdbus-1-3 libxcb-xinerama0 libxcb-cursor0 libxcb-shape0 \
29+
libxcb-icccm4 libxcb-keysyms1 libxcb-render-util0 libxcb-image0 \
30+
libnss3
31+
32+
- name: Setup pnpm
33+
uses: pnpm/action-setup@v4
34+
with:
35+
version: 10
36+
37+
- name: Setup Node.js
38+
uses: actions/setup-node@v4
39+
with:
40+
node-version: 20
41+
cache: 'pnpm'
42+
43+
- name: Install Node dependencies
44+
run: pnpm install --frozen-lockfile
45+
46+
- name: Setup uv
47+
uses: astral-sh/setup-uv@v4
48+
with:
49+
enable-cache: true
50+
cache-dependency-glob: 'uv.lock'
51+
52+
- name: Setup Python environment
53+
run: |
54+
uv python install 3.12
55+
uv venv --python 3.12 .venv
56+
uv sync
57+
58+
- name: Version consistency
59+
run: node scripts/version.mjs check
60+
61+
- name: Lint
62+
run: pnpm run lint
63+
64+
- name: Typecheck
65+
run: pnpm run typecheck
66+
67+
- name: Backend tests
68+
# test_transcription is excluded — it downloads a whisper model.
69+
# Clipboard tests self-skip headless (no DISPLAY/WAYLAND_DISPLAY).
70+
run: uv run -p .venv pytest src-pyloid/tests/ -q --ignore=src-pyloid/tests/test_transcription.py
71+
72+
# Build verification without installers — catches PyInstaller spec drift
73+
# (new modules/data files not bundled) before it surfaces on a release tag.
74+
# Runs in parallel with `test` so test feedback isn't delayed. Free on
75+
# public repos; no artifacts uploaded (dist/ is hundreds of MB and the
76+
# release workflow's workflow_dispatch covers "build me this branch").
77+
build:
78+
runs-on: ubuntu-22.04
79+
timeout-minutes: 25
80+
env:
81+
QT_QPA_PLATFORM: offscreen
82+
steps:
83+
- uses: actions/checkout@v4
84+
85+
- name: Install system dependencies
86+
run: |
87+
sudo apt-get update
88+
sudo apt-get install -y \
89+
libgl1-mesa-dev libegl1-mesa-dev libxkbcommon0 libfontconfig1 \
90+
libdbus-1-3 libxcb-xinerama0 libxcb-cursor0 libxcb-shape0 \
91+
libxcb-icccm4 libxcb-keysyms1 libxcb-render-util0 libxcb-image0 \
92+
libnss3 libasound2
93+
94+
- name: Setup pnpm
95+
uses: pnpm/action-setup@v4
96+
with:
97+
version: 10
98+
99+
- name: Setup Node.js
100+
uses: actions/setup-node@v4
101+
with:
102+
node-version: 20
103+
cache: 'pnpm'
104+
105+
- name: Install Node dependencies
106+
run: pnpm install --frozen-lockfile
107+
108+
- name: Setup uv
109+
uses: astral-sh/setup-uv@v4
110+
with:
111+
enable-cache: true
112+
cache-dependency-glob: 'uv.lock'
113+
114+
- name: Setup Python environment
115+
run: |
116+
uv python install 3.12
117+
uv venv --python 3.12 .venv
118+
uv sync
119+
120+
- name: Build application
121+
run: pnpm run build
122+
123+
- name: Smoke test - binary exists and shared libraries resolve
124+
run: |
125+
test -x ./dist/VoiceFlow/VoiceFlow || { echo "::error::dist/VoiceFlow/VoiceFlow missing or not executable"; exit 1; }
126+
MISSING=$(ldd ./dist/VoiceFlow/VoiceFlow 2>&1 | grep "not found" || true)
127+
if [ -n "$MISSING" ]; then
128+
echo "::error::Missing shared libraries:"
129+
echo "$MISSING"
130+
exit 1
131+
fi
132+
echo "Build OK: $(du -sh dist/VoiceFlow | cut -f1) bundle, all shared libraries resolved."

‎.github/workflows/release.yml‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -75,6 +75,19 @@ jobs:
7575
fi
7676
echo "version=$VERSION" >> "$GITHUB_OUTPUT"
7777
78+
- name: Verify version consistency
79+
run: |
80+
node scripts/version.mjs check
81+
# On tag builds, the tag must match the committed version — catches
82+
# tagging the wrong commit before 30 minutes of build time is spent.
83+
if [[ "${{ github.event_name }}" == "push" ]]; then
84+
PKG=$(node -p "require('./package.json').version")
85+
if [[ "$PKG" != "${{ steps.version.outputs.version }}" ]]; then
86+
echo "::error::Tag v${{ steps.version.outputs.version }} does not match package.json version $PKG — bump with scripts/version.mjs and re-tag."
87+
exit 1
88+
fi
89+
fi
90+
7891
- name: Build application
7992
run: pnpm run build
8093

@@ -98,6 +111,24 @@ jobs:
98111
rm -f ./dist/VoiceFlow/_internal/libasound*
99112
echo "Removed bundled PortAudio + ALSA (will use system libraries at runtime)"
100113
114+
- name: Verify audio library layout
115+
run: |
116+
# Regression guard for the recurring "no mics in release build" bug
117+
# (v1.3.x–v1.5.0): the bundled libasound/libportaudio MUST be gone...
118+
LEFTOVER=$(find ./dist/VoiceFlow/_internal -maxdepth 1 \( -name 'libasound*' -o -name 'libportaudio*' \) | head -5)
119+
if [ -n "$LEFTOVER" ]; then
120+
echo "::error::Bundled audio libraries still present (would break mic enumeration on non-Ubuntu distros): $LEFTOVER"
121+
exit 1
122+
fi
123+
# ...while PyAV's renamed copy MUST remain — libavdevice lists
124+
# libasound-cfbebb71.so.2.0.0 in NEEDED and the app crashes at
125+
# startup without it (different SONAME, coexists with system lib).
126+
if ! ls ./dist/VoiceFlow/_internal/av.libs/libasound-* >/dev/null 2>&1; then
127+
echo "::error::PyAV's av.libs/libasound-* is missing — startup will fail with ImportError. The cleanup step deleted too much."
128+
exit 1
129+
fi
130+
echo "Audio library layout OK (system libasound/portaudio at runtime, PyAV copy intact)"
131+
101132
- name: Clear executable stack flags
102133
run: |
103134
# python-build-standalone builds ship libpython with GNU_STACK RWE,
@@ -183,6 +214,18 @@ jobs:
183214
fi
184215
echo "version=$VERSION" >> "$GITHUB_OUTPUT"
185216
217+
- name: Verify version consistency
218+
shell: bash
219+
run: |
220+
node scripts/version.mjs check
221+
if [[ "${{ github.event_name }}" == "push" ]]; then
222+
PKG=$(node -p "require('./package.json').version")
223+
if [[ "$PKG" != "${{ steps.version.outputs.version }}" ]]; then
224+
echo "::error::Tag v${{ steps.version.outputs.version }} does not match package.json version $PKG — bump with scripts/version.mjs and re-tag."
225+
exit 1
226+
fi
227+
fi
228+
186229
- name: Build application
187230
run: pnpm run build
188231

‎CLAUDE.md‎

Lines changed: 65 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,8 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
66

77
VoiceFlow is a cross-platform voice-to-text paste utility built with Pyloid (Python desktop framework using PySide6/Qt WebEngine) and React. Users hold a hotkey to record audio, release to transcribe using faster-whisper, and the text is automatically pasted at the cursor. Supports Windows, Linux (Wayland/X11), and macOS.
88

9+
As of v1.6.0, VoiceFlow also ships a **Meeting Mode** — long-form recording with mic + system-loopback capture, async transcription, and LLM-generated summaries. This is a separate feature surface from the push-to-talk paste flow, kept self-contained under `services/recording/` so it can evolve (or be extracted) without touching PTT code.
10+
911
## Commands
1012

1113
```bash
@@ -26,28 +28,44 @@ pnpm run build:installer # Windows (.exe via Inno Setup)
2628
pnpm run build:installer:linux # Linux (.tar.gz + .AppImage)
2729
pnpm run build:installer:macos # macOS (.dmg)
2830

29-
# Run Python tests
30-
cd VoiceFlow && uv run -p .venv pytest src-pyloid/tests/
31+
# Run Python tests (fast suite — excludes test_transcription, which downloads a model)
32+
pnpm run test
33+
pnpm run test:all # includes test_transcription
3134

3235
# Run single test file
3336
uv run -p .venv pytest src-pyloid/tests/test_transcription.py -v
3437

38+
# Everything CI runs (lint + typecheck + tests + version consistency)
39+
pnpm run check
40+
41+
# Version bump — updates all five version files (package.json, voiceflow.iss,
42+
# pyproject.toml, constants.ts, uv.lock); never edit them by hand
43+
pnpm run version:bump X.Y.Z
44+
pnpm run version:check
45+
3546
# Run frontend only (for UI development)
3647
pnpm run vite
3748

38-
# Lint frontend
49+
# Lint / typecheck frontend
3950
pnpm run lint
51+
pnpm run typecheck
4052
```
4153

54+
CI (`.github/workflows/ci.yml`) runs lint, typecheck, version check, and the fast
55+
test suite on every PR and push to main. The release workflow
56+
(`.github/workflows/release.yml`) additionally refuses tags that don't match
57+
package.json and verifies the Linux artifact's audio-library layout
58+
(bundled libasound/libportaudio removed, PyAV's av.libs copy intact).
59+
4260
## Architecture
4361

4462
### Backend (src-pyloid/)
4563

4664
Python backend using Pyloid framework with PySide6:
4765

4866
- **main.py** - Application entry point. Creates Pyloid app, tray icon, main dashboard window, and recording popup window. Sets up UI callbacks connecting backend events to popup state changes.
49-
- **server.py** - RPC server using `PyloidRPC`. Exposes methods (`get_settings`, `update_settings`, `get_history`, etc.) that frontend calls via `pyloid-js` RPC.
50-
- **app_controller.py** - Singleton controller orchestrating all services. Handles hotkey activate/deactivate flow: start recording -> stop recording -> transcribe -> paste at cursor -> save to history.
67+
- **server.py** - RPC server using `PyloidRPC`. Exposes PTT methods (`get_settings`, `update_settings`, `get_history`, etc.) plus the full Meeting Mode surface (`meetings.list_audio_sources`, `meetings.start`, `meetings.pause`, `meetings.resume`, `meetings.stop`, `meetings.transcribe`, `meetings.summarize`, `meetings.get_llm_config`, `meetings.test_llm_connection`, etc.) that frontend calls via `pyloid-js` RPC.
68+
- **app_controller.py** - Singleton controller orchestrating all services. Handles hotkey activate/deactivate flow: start recording -> stop recording -> transcribe -> paste at cursor -> save to history. Also constructs and owns the `MeetingsController` (exposed as `controller.meetings`) and runs an unfinished-recording recovery sweep on startup.
5169

5270
**Services (src-pyloid/services/):**
5371
- `audio.py` - Microphone recording using sounddevice, streams amplitude for visualizer
@@ -56,21 +74,37 @@ Python backend using Pyloid framework with PySide6:
5674
- `clipboard.py` - Clipboard operations and paste-at-cursor using pyautogui
5775
- `settings.py` - Settings management with defaults
5876
- `database.py` - SQLite database for settings and history (stored at ~/.VoiceFlow/VoiceFlow.db)
59-
- `logger.py` - Domain-based logging with hybrid format `[timestamp] [LEVEL] [domain] message | {json}`. Supports domains: model, audio, hotkey, settings, database, clipboard, window. Configured with 100MB log rotation.
60-
- `model_manager.py` - Whisper model download/cache management using huggingface_hub. Provides download progress tracking (percent, speed, ETA), cancellation via CancelToken, daemon thread execution, and `clear_cache()` to delete only VoiceFlow's faster-whisper models.
77+
- `logger.py` - Domain-based logging with hybrid format `[timestamp] [LEVEL] [domain] message | {json}`. Supports domains: model, audio, hotkey, settings, database, clipboard, window, plus Meeting-Mode domains (recording, transcribe, summary, llm). Configured with 100MB log rotation.
78+
- `model_catalog.py` - Single source of truth for the Whisper model catalog (names, sizes, HF repo IDs). Imported by `model_manager`, `transcription`, and `settings`; the frontend gets repo IDs over RPC (`get_model_info` → `repoId`) rather than keeping its own copy. Never re-declare the model list elsewhere.
79+
- `model_manager.py` - Whisper model download/cache management using huggingface_hub. Owns the single-flight background download session (`start_download` / `cancel_download` / `get_download_status` — the RPC surface), download progress tracking (percent, speed, ETA), cancellation via CancelToken, and `delete_model()` / `clear_cache()`. Model *loading* lives in `TranscriptionService` (the only load path, since it resolves the user's device preference).
80+
81+
**Meeting Mode services (src-pyloid/services/recording/):**
82+
Self-contained per `docs/adr/0003-meeting-mode-isolation.md`; do not call these from the PTT path and vice versa.
83+
- `controller.py` - `MeetingsController` — the feature's facade. All RPC handlers go through this object. Emits `recording-state`, `recording-transcribe-progress`, and `recording-summarize-progress` events to the frontend via the emitter installed by `main.py`.
84+
- `recorder.py` - Long-form recorder with pause/resume, segmented WAV writing, and clock tracking. Sources are fixed at `start()` and cannot change mid-recording.
85+
- `audio_source.py` - Enumerates available mic + loopback devices for the UI device picker.
86+
- `loopback_linux.py` / `loopback_pulse.py` / `loopback_windows.py` - Platform-specific system-audio capture (PulseAudio/PipeWire on Linux, WASAPI loopback on Windows).
87+
- `clock.py` - Monotonic recording clock that survives pause/resume.
88+
- `llm.py` - LLM client with preset + custom-endpoint support; used by summary/title generation.
89+
- `summary.py` / `title.py` - LLM-driven summary and auto-title generation for finished recordings.
90+
- `secrets.py` - API-key storage for LLM providers (kept out of the main settings table).
91+
- `export.py` - Exports a recording's transcript/summary to text formats.
92+
- `recovery.py` - On startup, sweeps recordings left in `recording` / `paused` state from a previous (crashed) session and rolls them forward.
93+
- `audio_scheme.py` / `audio_scheme_handler.py` - Custom Qt `audio://` URL scheme so the WebEngine `<audio>` element can stream recording WAVs from disk without a server.
6194

6295
### Frontend (src/)
6396

6497
React 18 + TypeScript + Vite frontend:
6598

6699
- **App.tsx** - Hash-based routing between `/popup`, `/onboarding`, and `/dashboard`. Checks model cache on startup and shows recovery modal if model is missing.
67-
- **lib/api.ts** - RPC wrapper using `pyloid-js` to call Python backend methods. Includes model management APIs (`getModelInfo`, `startModelDownload`, `cancelModelDownload`).
68-
- **lib/types.ts** - TypeScript interfaces for Settings, HistoryEntry, Stats, Options, ModelInfo, DownloadProgress
69-
- **pages/** - Popup (recording indicator), Onboarding (includes model download step), Dashboard
100+
- **lib/api.ts** - RPC wrapper using `pyloid-js` to call Python backend methods. Includes model management APIs (`getModelInfo`, `startModelDownload`, `cancelModelDownload`) plus the full `recordings*` / meetings RPC surface.
101+
- **lib/types.ts** - TypeScript interfaces for Settings, HistoryEntry, Stats, Options, ModelInfo, DownloadProgress, plus meeting types: `Recording`, `RecordingSegment`, `RecorderState`, and LLM config types.
102+
- **pages/** - Popup (recording indicator), Onboarding (includes model download step), Dashboard. Dashboard uses React Router for sub-routes: `history`, `meetings`, `meetings/record`, `meetings/:id`, `settings`.
70103
- **components/** - Feature components plus shadcn/ui components in `components/ui/`
71104
- `ModelDownloadProgress.tsx` - Download progress UI with progress bar, speed, ETA, and retry support
72105
- `ModelDownloadModal.tsx` - Dialog wrapper for model downloads triggered from settings
73106
- `ModelRecoveryModal.tsx` - Startup modal for missing model recovery
107+
- `meetings/` - Meeting Mode UI: `MeetingsListPage`, `MeetingRecorderPage`, `MeetingDetailPage`, `MeetingImportDialog`, `MeetingRecorderContext` (cross-route recorder state), `AudioPlayer`, `LevelMeter`, `StatusLine`, `TranscriptView`, `SummaryView`, `RetranscribeDialog`, `LLMSettingsSection`, `MeetingsSettingsSection`.
74108

75109
### Frontend-Backend Communication
76110

@@ -124,6 +158,27 @@ For transparent popup windows on Windows:
124158
6. On completion, model is cached in huggingface cache directory
125159
7. Turbo model uses `mobiuslabsgmbh/faster-whisper-large-v3-turbo` (same as faster-whisper internal mapping)
126160

161+
### Meeting Mode (long-form recording)
162+
163+
Separate from the PTT paste flow. Entrypoint: `controller.meetings` (`MeetingsController`). See `docs/adr/0001-stereo-channel-layout-for-recordings.md` for the on-disk audio layout decision.
164+
165+
1. UI calls `meetings.list_audio_sources()` to populate the device picker (mic + loopback).
166+
2. `meetings.start(mic_device_id, loopback_device_id)` opens up to two simultaneous capture streams. Sources are fixed at start — they cannot be added/removed mid-recording.
167+
3. Audio is written to a WAV file under `~/.VoiceFlow/recordings/`:
168+
- Two active sources → **stereo 16 kHz PCM16**, mic on **L**, loopback on **R** (kept separate on purpose; enables future speaker diarization with no ML — see ADR 0001).
169+
- One active source → mono 16 kHz PCM16.
170+
4. `meetings.pause()` / `meetings.resume()` use a monotonic `Clock` to track real recording time; segments are stitched into one logical recording.
171+
5. `meetings.stop()` finalizes the WAV and persists metadata. Recording rows live in the same SQLite DB but in their own table.
172+
6. Transcription is **async and on-demand**: `meetings.transcribe(id)` runs faster-whisper in a daemon thread and emits `recording-transcribe-progress` events. Long jobs do not block the RPC channel (see fix `dc04d29`).
173+
7. After transcription, `meetings.summarize(id, prompt)` calls the configured LLM provider (preset or custom endpoint) to produce an AI summary, and `title.py` auto-generates a title. LLM config and API keys live in `services/recording/llm.py` + `secrets.py`, not in the main `settings` table.
174+
8. Audio playback in the detail page uses a custom Qt `audio://` URL scheme (`audio_scheme.py`) so the WebEngine can stream the WAV directly without an HTTP server.
175+
9. On startup, `recovery.py` rolls forward any recordings left in `recording` / `paused` state from a crashed previous session.
176+
177+
**Platform quirks**:
178+
- Linux loopback uses PulseAudio/PipeWire monitor sources (`loopback_pulse.py` / `loopback_linux.py`).
179+
- Windows loopback uses WASAPI. Must open the loopback stream at the device's native channel count (`max_output_channels`) — opening at a forced channel count fails on many devices (fixes `96b0b73`, `13d45be`).
180+
- Pyloid validates window IDs on every RPC roundtrip from a background thread; long-running meeting RPCs work around this (see `135fdd7`).
181+
127182
## Key Patterns
128183

129184
- **Singleton controller**: `get_controller()` returns singleton `AppController` instance

‎docs/adr/0001-stereo-channel-layout-for-recordings.md‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
# Stereo channel layout for two-source Recordings
2+
3+
When a Recording captures both a mic and a system-loopback source, the WAV is
4+
written as stereo 16 kHz PCM16 with the **mic on the left channel (ch 0)** and
5+
the **loopback on the right channel (ch 1)** — deliberately *not* mixed down to
6+
mono. Keeping the sides separate means "who said what" (you vs. them) is
7+
recoverable later by channel, enabling speaker attribution with zero ML
8+
diarization. A single-source Recording is mono.
9+
10+
## Consequences
11+
12+
- Sources are fixed at `start()`; a source cannot be added or removed
13+
mid-recording, because the channel count of the WAV is decided up front.
14+
- In stereo mode the recorder pads a starved side with silence (starvation
15+
detection in `recorder.py`) so the channels stay time-aligned — alignment is
16+
the property that makes per-channel attribution trustworthy.
17+
- Playback of a stereo Recording sounds hard-panned (you fully left, them
18+
fully right). That is accepted, not a bug.

0 commit comments

Comments
 (0)