Self-hosted, private, bilingual AI English tutor — voice-in, voice-out, built on Open WebUI, Whisper, Groq, and Kokoro TTS.
Build my own AI English tutor. The existing apps didn't quite fit what I actually needed: the good ones were either paid subscriptions with limited trials, restricted how much you could practice per day, or simply weren't built for bilingual practice — moving naturally between Russian and English in the same conversation, the way I actually think and speak while I'm still building fluency. What I built instead is free, private, bilingual, and has no usage caps beyond what my own server and API quotas allow.
The system is a voice-in, voice-out pipeline running entirely on my own VPS, with a single external call to a free LLM API:
My voice → Speech-to-Text (local) → LLM (API call) → Text-to-Speech (local) → Spoken reply
Everything except the LLM inference call happens on my own server. No audio ever leaves my machine for transcription, and no synthesized speech is generated by a third party.
Platform: Open WebUI Self-hosted chat interface (Docker container on my VPS) that ties together the STT engine, the LLM connection, and the TTS engine, plus lets me define a custom "persona" with its own system prompt.
Speech-to-Text: faster-whisper (Whisper base model)
Runs locally inside the Open WebUI container, so nothing is sent to a third-party speech API. Language detection is left on automatic (the language field is simply left blank), so it picks up Russian and English in the same session without me having to switch settings manually.
LLM: Groq API, Llama 3.1 8B Instant The only component that isn't self-hosted, since running a large model locally isn't realistic on a CPU-only VPS. I switched to the smaller 8B model after running into daily rate limits during heavy testing — it has a much higher daily token quota than the 70B model, and Groq's free tier still covers it comfortably for personal daily practice. Zero Data Retention (both the global toggle and the inference-API-specific one) is enabled in the Groq console, so inputs and outputs aren't logged, not even temporarily.
Text-to-Speech: kokoro-fastapi (self-hosted)
A separate Docker container (ghcr.io/remsky/kokoro-fastapi-cpu) running the Kokoro TTS model, exposed as an OpenAI-compatible endpoint that Open WebUI talks to over the internal Docker network. Speech synthesis happens entirely on my own server. Since kokoro-fastapi's own test interface has no built-in login, I moved it behind its own subdomain, kokoro.lilia.am, with HTTPS and Basic Auth at the Nginx level — closing off what had briefly been an open, unauthenticated endpoint.
The coach persona A custom system prompt defines a bilingual speaking coach: it understands Russian, broken English, or fluent English in any mix, responds in kind, offers a more natural or advanced way to phrase things when useful, and keeps its own replies short (1 to 3 sentences) to keep the pace of the conversation natural and easy to follow. The coach always replies out loud in English, since that's the language being practiced.
None of this worked cleanly on the first try, and the troubleshooting was arguably the most educational part.
The built-in browser TTS was silently broken. Open WebUI ships with an in-browser Kokoro.js option that looked like the obvious choice, until I noticed the selected voice never actually changed the output. Digging into the browser's DevTools (Console and Network tabs) confirmed it: no request was ever being sent to the voice engine, and the browser was silently falling back to the OS's default voice. This turned out to be a known, unresolved issue in the current Open WebUI release. The fix was to abandon the browser-based approach entirely and run the TTS engine as its own server-side container instead.
Docker containers couldn't reach each other via the "obvious" address. Once kokoro-fastapi was running, Open WebUI still couldn't connect to it through the Docker bridge gateway IP (172.17.0.1); a Connect call failed error showed up in the container logs. Switching to the target container's own internal IP address (found via docker inspect) resolved it immediately.
Nginx was cutting off longer responses. Short replies worked fine, but anything requiring more than about a minute of speech synthesis failed with a 504 Gateway Timeout. Nginx's default proxy_read_timeout is 60 seconds, and the reverse proxy in front of Open WebUI simply gave up before the TTS generation finished. Raising proxy_read_timeout, proxy_connect_timeout, and proxy_send_timeout to 120 seconds fixed it.
| Component | Where it runs | Data leaves the server? | Cost |
|---|---|---|---|
| Speech-to-text | My VPS (local Whisper) | No | Free |
| LLM inference | Groq API | Yes, but Zero Data Retention enabled | Free (generous daily token limit) |
| Text-to-speech | My VPS (kokoro-fastapi) | No | Free |
| Chat history, knowledge base | My VPS (Open WebUI database) | No | Free |
The only point where anything leaves my own infrastructure is the LLM call itself, and even that is covered by Groq's Zero Data Retention setting: no logging of inputs or outputs, even temporarily.
During development, I discovered an additional useful benefit. Whisper doesn't just transcribe: it's an acoustic model, so when my pronunciation is off, it sometimes mishears the word entirely and transcribes something that sounds similar but means something different. Because the coach persona sees the transcribed text, it naturally catches these mismatches as if they were genuine errors, corrects them, and asks me to repeat the phrase. This surfaces pronunciation feedback through the mechanics of general-purpose speech recognition, not through a dedicated phonetic-scoring model.
Worth being precise about what this is and isn't: this isn't the same as a purpose-built pronunciation app, which typically scores pronunciation at the phoneme level against an acoustic reference. What I have is an extra, useful signal in practice, even though it's cruder than a dedicated scoring model.
The tutor runs at a live URL and works from both desktop and mobile browsers, including Open WebUI's Call mode for a hands-free voice conversation on the go (it works both with and without headphones, though longer sessions in Call mode do use up more of the daily token budget, simply because the conversations tend to run longer).
This is a personal server running on my own domain, chat.lilia.am.
Whisper (STT) technically supports Armenian (it's among the roughly 100 languages the model covers), but recognition quality is noticeably lower than for English or Russian. kokoro-fastapi's voice output doesn't support Armenian: the available languages are English, Spanish, French, Hindi, Italian, Japanese, Brazilian Portuguese, and Chinese.