[Feature request] Listening for wake word

[shenzo1]

Thank you for this great project. Would there be a way to have voicemode listen for a wake word before activating recording and transcription so that Claude Code can always be listening? I have tried longer listening time but that tends to result in having to restart the mcp server. I am trying to use voicemode to be able to call Claude Code via livekit and Sip telephony.

[mbailey]

By chance I spiked this last night. Will push the branch now but as this is unpaid open source will need to wait till weekend at least for full implementation.

The implementation requires whisper.cpp running locally (using whisper-stream). The branch is "let-me-finish" because I was trying accomodate situations where pauses won't trigger silence detection - you say "I'm done" (or custom safeword).

It was super exciting! Wakeword will be pretty easy to add as well.

Would you like to beta test?

[shenzo1]

That's awesome! Yes would love to beta test!

[shenzo1]

I am stuck at trying to install whisper-stream from the whisper.cpp example.

I have installed opuslib but still get an error.

(.venv) (base) shenzo@M1 voicemode % voicemode converse --uninterrupted
/opt/homebrew/lib/python3.11/site-packages/pydub/utils.py:14: DeprecationWarning: 'audioop' is deprecated and slated for removal in Python 3.13
import audioop
/opt/homebrew/lib/python3.11/site-packages/pkg_resources/init.py:3146: DeprecationWarning: Deprecated call to pkg_resources.declare_namespace('pyannote').
Implementing implicit namespace packages (as specified in PEP 420) is preferred to pkg_resources.declare_namespace. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
2025-09-12 00:18:13,648 - asyncio - ERROR - Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x122236f90>
2025-09-12 00:18:13,649 - asyncio - ERROR - Unclosed connector
connections: ['deque([(<aiohttp.client_proto.ResponseHandler object at 0x1223ca350>, 3096830.453658166)])']
connector: <aiohttp.connector.TCPConnector object at 0x1222376d0>
2025-09-12 00:18:13,658 - voice_mode.pronounce - INFO - Loaded pronunciation rules from /Users/shenzo/voicemode/voice_mode/data/default_pronunciation.yaml
2025-09-12 00:18:13,666 - voicemode - INFO - TTS: Converting text to speech: 'Hello! How can I help you today?'
2025-09-12 00:18:13,666 - voicemode - INFO - TTS Request Details:
2025-09-12 00:18:13,666 - voicemode - INFO - • Endpoint URL: http://127.0.0.1:8880/v1
2025-09-12 00:18:13,666 - voicemode - INFO - • Model: tts-1
2025-09-12 00:18:13,666 - voicemode - INFO - • Voice: af_alloy
2025-09-12 00:18:13,666 - voicemode - INFO - • Format: default
2025-09-12 00:18:13,666 - voicemode - INFO - • Client Key: tts
2025-09-12 00:18:13,666 - voicemode - INFO - • Detected Provider: kokoro (based on URL: http://127.0.0.1:8880/v1)
2025-09-12 00:18:13,666 - voicemode - INFO - Using streaming with pcm format
2025-09-12 00:18:13,666 - voicemode - INFO - Using streaming playback for pcm
2025-09-12 00:18:13,667 - voicemode - INFO - opuslib not available - Opus streaming will use fallback method
2025-09-12 00:18:13,667 - voicemode - INFO - Starting streaming TTS with format: pcm
2025-09-12 00:18:13,772 - voicemode - INFO - Starting true HTTP streaming with iter_bytes()
2025-09-12 00:18:45,036 - voicemode - INFO - First audio chunk received after 31.367s
2025-09-12 00:18:47,424 - voicemode - INFO - TTFA (first chunk): 31.367s
2025-09-12 00:18:47,434 - voicemode - INFO - Streaming complete - TTFA: 31.367s, Total: 33.752s, Chunks: 26
2025-09-12 00:18:47,457 - voicemode - INFO - ✓ TTS streamed successfully - TTFA: 31.367s
2025-09-12 00:18:48,802 - voice-mode - ERROR - whisper-stream not found in PATH. Please install whisper-stream.
Error: whisper-stream not available for uninterrupted mode
(.venv) (base) shenzo@M1 voicemode % pip install opuslib
Requirement already satisfied: opuslib in ./.venv/lib/python3.13/site-packages (3.0.1)

[mbailey]

Ah, it's not even in Alpha yet!

Last night, I freaked myself out a bit by adding support for "Hey Claude!" And having Siri's voice respond with the current time.

It was late and I was just playing around before bed. It's a lot more work to make it work for everyone. That could eat my whole weekend so I can't guarantee any timeline, but I'm pretty excited about having it.

[shenzo1]

Thanks for your effort. Sounds very cool!

[shenzo1]

Actually I finally managed to install whisper-stream, I was building in the wrong folder instead of the whisper.cpp folder.

Now I seem to have everything installed but it quits just after the tts. BUt I know it is pre-alpha :) so I'll just wait that you have a beta to test.

(.venv) (base) shenzo@M1 voicemode % voicemode converse --uninterrupted
/opt/homebrew/lib/python3.11/site-packages/pydub/utils.py:14: DeprecationWarning: 'audioop' is deprecated and slated for removal in Python 3.13
import audioop
/opt/homebrew/lib/python3.11/site-packages/pkg_resources/init.py:3146: DeprecationWarning: Deprecated call to pkg_resources.declare_namespace('pyannote').
Implementing implicit namespace packages (as specified in PEP 420) is preferred to pkg_resources.declare_namespace. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
2025-09-12 22:24:34,579 - asyncio - ERROR - Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x126cac6d0>
2025-09-12 22:24:34,579 - asyncio - ERROR - Unclosed connector
connections: ['deque([(<aiohttp.client_proto.ResponseHandler object at 0x1271ca350>, 3176411.145936625)])']
connector: <aiohttp.connector.TCPConnector object at 0x126e662d0>
2025-09-12 22:24:34,597 - voice_mode.pronounce - INFO - Loaded pronunciation rules from /Users/shenzo/voicemode/voice_mode/data/default_pronunciation.yaml
2025-09-12 22:24:34,607 - voicemode - INFO - TTS: Converting text to speech: 'Hello! How can I help you today?'
2025-09-12 22:24:34,607 - voicemode - INFO - TTS Request Details:
2025-09-12 22:24:34,607 - voicemode - INFO - • Endpoint URL: http://127.0.0.1:8880/v1
2025-09-12 22:24:34,607 - voicemode - INFO - • Model: tts-1
2025-09-12 22:24:34,607 - voicemode - INFO - • Voice: af_sky
2025-09-12 22:24:34,607 - voicemode - INFO - • Format: default
2025-09-12 22:24:34,608 - voicemode - INFO - • Client Key: tts
2025-09-12 22:24:34,608 - voicemode - INFO - • Detected Provider: kokoro (based on URL: http://127.0.0.1:8880/v1)
2025-09-12 22:24:34,608 - voicemode - INFO - Using streaming with pcm format
2025-09-12 22:24:34,608 - voicemode - INFO - Using streaming playback for pcm
2025-09-12 22:24:34,622 - voicemode - INFO - Starting streaming TTS with format: pcm
2025-09-12 22:24:34,677 - voicemode - INFO - Starting true HTTP streaming with iter_bytes()
2025-09-12 22:24:36,089 - voicemode - INFO - First audio chunk received after 1.466s
2025-09-12 22:24:38,310 - voicemode - INFO - TTFA (first chunk): 1.466s
2025-09-12 22:24:38,310 - voicemode - INFO - Streaming complete - TTFA: 1.466s, Total: 3.687s, Chunks: 25
2025-09-12 22:24:38,313 - voicemode - INFO - ✓ TTS streamed successfully - TTFA: 1.466s
No speech detected | Timing: ttfa 1.5s, gen 1.5s, play 2.2s, record 1.6s, stt 0.0s, total 5.4s

[mbailey]

Make sure whisper-stream is in your $PATH.

Try using whisper-stream from the command line first to make sure it's working.

The following command worked fine for me just now (note, this is "end word", not "wake word":

(feature/let-me-finish stash:2 10 days ago)$ uv run voicemode converse -u --end-phrase finished -m "hello there"
2025-09-13 20:08:16,012 - asyncio - ERROR - Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x111f2bdd0>
2025-09-13 20:08:16,012 - asyncio - ERROR - Unclosed connector
connections: ['deque([(<aiohttp.client_proto.ResponseHandler object at 0x11214f000>, 23702.199287291)])']
connector: <aiohttp.connector.TCPConnector object at 0x111ea4510>
2025-09-13 20:08:16,017 - voice_mode.pronounce - INFO - Loaded pronunciation rules from /Users/admin/Code/github.com/mbailey/voicemode-let-me-finish/voice_mode/data/default_pronunciation.yaml
2025-09-13 20:08:16,025 - voice_mode.pronounce - INFO - Loaded pronunciation rules from /Users/admin/.voicemode/config/pronunciation.yaml
2025-09-13 20:08:16,034 - voicemode - INFO - TTS: Converting text to speech: 'hello there'
2025-09-13 20:08:16,034 - voicemode - INFO - TTS Request Details:
2025-09-13 20:08:16,034 - voicemode - INFO -   • Endpoint URL: http://127.0.0.1:8880/v1
2025-09-13 20:08:16,034 - voicemode - INFO -   • Model: tts-1
2025-09-13 20:08:16,034 - voicemode - INFO -   • Voice: af_alloy
2025-09-13 20:08:16,034 - voicemode - INFO -   • Format: default
2025-09-13 20:08:16,034 - voicemode - INFO -   • Client Key: tts
2025-09-13 20:08:16,034 - voicemode - INFO -   • Detected Provider: kokoro (based on URL: http://127.0.0.1:8880/v1)
2025-09-13 20:08:16,034 - voicemode - INFO - Using streaming with pcm format
2025-09-13 20:08:16,034 - voicemode - INFO - Using streaming playback for pcm
2025-09-13 20:08:16,034 - voicemode - INFO - opuslib not available - Opus streaming will use fallback method
2025-09-13 20:08:16,034 - voicemode - INFO - Starting streaming TTS with format: pcm
2025-09-13 20:08:16,053 - voicemode - INFO - Starting true HTTP streaming with iter_bytes()
2025-09-13 20:08:16,402 - voicemode - INFO - First audio chunk received after 0.368s
2025-09-13 20:08:17,881 - voicemode - INFO - TTFA (first chunk): 0.368s
2025-09-13 20:08:17,882 - voicemode - INFO - Streaming complete - TTFA: 0.368s, Total: 1.847s, Chunks: 16
2025-09-13 20:08:17,885 - voicemode - INFO - TTS audio saved to: /Users/admin/.voicemode/audio/2025/09/20250913_200817_884_pbpf23_tts.wav
2025-09-13 20:08:17,887 - voicemode - INFO - ✓ TTS streamed successfully - TTFA: 0.368s

📢 Spoke: hello there
🎤 Heard: [00:00:00.000 --> 00:00:02.560]   Hi there. [00:00:02.560 --> 00:00:06.560]   How are things going? [00:00:00.000 --> 00:00:00.500]   Hi there. [00:00:00.500 --> 00:00:07.200]   How are things going? Now I'm finished.
⏱️  Timing: ttfa 0.4s, gen 0.4s, play 1.5s, record 9.7s, stt 0.0s, total 11.5s

[mbailey]

I'm currently working on creating The VoiceMode Podcast so don't know if I'll progress end / wake words this weekend. Feel free to play with what I've shared on let-me-finish branch if you like.

[kindcreator]

@shenzo1
@mbailey any updates on wake word topic?

I do also need that a lot

If there is some work I can do to speedup the process I would try to allocate some time @mbailey
let me know plz

[dmora]

Hey @mbailey - I explored the feature/let-me-finish branch and love the command routing architecture!

One architectural thought on wake word detection:

Current approach (whisper-stream continuous):

• ~15-40% CPU continuous
• Flexible - any phrase works
• Battery drain for always-on use

Hybrid proposal:

Stage 1: openWakeWord (2-5% CPU, always-on)
    ↓ wake word detected  
Stage 2: whisper-stream (on-demand transcription)
    ↓ fuzzy routing (your existing code)

This mirrors how Alexa/Siri work - dedicated low-power wake word detection, then full ASR only after activation.

openWakeWord is MIT licensed, cross-platform (Python/ONNX), and has pre-trained models.

Happy to contribute this if you think it's the right direction. The existing command router and fuzzy matching would stay as-is - just adding an efficient front-end.

Thoughts?

[mbailey]

That would be a very welcome contribution @dmora !

@ai-cora and I spent some time on this over the weekend but failed to get matching to work. I then had the idea of adding a Siri Shortcut ("Hey Siri, get Cora") and that works well to spawn or interrupt her. It's great but having a free portable and private solution would better match the values of the project.

Have you been able to get it working? We tried with the demo words and didn't have any success but also did the "try, fail, try something else" so if you're willing to put in the time to investigate this could be a significant enhancement.