Personal text-to-speech webapp powered by VoxCPM2. Voice design, controllable cloning, ultimate cloning — Next.js on Vercel, GPU on Modal.

---

Features

• Voice Design — describe a voice in plain language (a young female, cheerful tone, normal pace) and synthesize from text
• Controllable Cloning — clone a speaker from a reference clip and override style (slightly faster, melancholy tone)
• Ultimate Cloning — highest fidelity clone using reference audio plus its transcript
• 30 languages auto-detected, no language tag required
• 48 kHz studio-quality output via AudioVAE V2
• Waveform player with scrubbing, play/pause, and download
• GPU warmup dialog with live status badge (red → amber → green) — no surprise cold starts mid-flow
• Voice presets persisted in localStorage
• Session history with click-to-replay

Model

Built on openbmb/VoxCPM2:

Property	Value
Parameters	2B
Backbone	MiniCPM-4
Architecture	Tokenizer-free, diffusion-autoregressive (LocEnc → TSLM → RALM → LocDiT)
Sample rate	48 kHz
Languages	30 (incl. Chinese dialects)
VRAM	~8 GB
RTF (RTX 4090)	~0.13 via nano-vllm-voxcpm
License	Apache-2.0

Stack

Layer	Tech
Frontend	Next.js 16 (App Router), TypeScript, Tailwind v4, shadcn/ui, JetBrains Mono
Audio UI	wavesurfer.js
Inference	nano-vllm-voxcpm 2.0.1, FastAPI
Runtime	Modal — serverless GPU, scale-to-zero
GPU	NVIDIA L4 (24 GB)
Build	CUDA 12.4.1, PyTorch 2.5.1+cu124, flash-attn 2.7.4.post1
Model storage	Baked into image at build time via huggingface_hub

Architecture

                                       ┌──────────────────────────────┐
                                       │   Modal (L4, scale-to-zero)  │
                                       │                              │
   Browser ──── fetch ─────────────────▶│   FastAPI ASGI               │
   Next.js / Vercel                     │   ├── GET  /health           │
                                        │   ├── POST /design           │
                                        │   ├── POST /clone-controllable
                                        │   └── POST /clone-ultimate   │
                                        │       │                      │
                                        │       ▼                      │
                                        │   nano-vllm-voxcpm           │
                                        │       │                      │
                                        │       ▼                      │
                                        │   VoxCPM2 (bf16)             │
                                        │   AudioVAE V2 → 48 kHz WAV   │
                                        └──────────────────────────────┘

Getting started

1. Deploy the Modal backend

pip install modal
modal token new          # one-time auth
cd modal && modal deploy app.py

First deploy pulls the CUDA image, installs flash-attn, and snapshots the VoxCPM2 weights into the image — takes a few minutes. Subsequent deploys are quick.

Modal prints the deployment URL on success, e.g.:

https://<user>--vox-populi-voxpopuli-fastapi-app.modal.run

2. Configure & run the frontend

cd web
cp .env.example .env.local
# edit .env.local — set NEXT_PUBLIC_MODAL_URL to your Modal URL
npm install
npm run dev

Open http://localhost:3000.

API reference

All endpoints return audio/wav on success. Errors return JSON {detail}.

Endpoint	Body (multipart unless noted)	Notes
GET /health	—	Lightweight warmup trigger. First call after idle blocks ~60s while VoxCPM2 loads.
POST /design	text, cfg_value? (2.0), inference_timesteps? (10)	Optional (description) prefix in text controls the generated voice.
POST /clone-controllable	text, reference_audio (file), cfg_value?, inference_timesteps?	Optional (style) prefix in text for tone/pace overrides.
POST /clone-ultimate	text, prompt_audio (file), prompt_text, cfg_value?, inference_timesteps?	prompt_text must be the exact transcript of prompt_audio.

Note: inference_timesteps is fixed at server boot in nano-vllm-voxcpm 2.0.1. The endpoint accepts it for forward-compat but per-request overrides are ignored.

Repo layout

vox-populi/
├── modal/                  # Modal Python app
│   ├── app.py              # nano-vllm-voxcpm + FastAPI
│   ├── requirements.txt
│   └── .python-version
├── web/                    # Next.js frontend
│   ├── app/
│   │   ├── page.tsx        # 3 modes, warmup dialog, advanced params
│   │   ├── layout.tsx
│   │   ├── globals.css
│   │   └── icon.svg
│   ├── components/
│   │   ├── ui/             # shadcn/ui primitives
│   │   └── studio/
│   │       └── player-bar.tsx  # wavesurfer.js player
│   └── .env.example
└── assets/
    └── banner.svg

Credits

• OpenBMB / VoxCPM2 — the model
• a710128 / nanovllm-voxcpm — the inference engine
• Modal — serverless GPU
• shadcn/ui — UI primitives
• wavesurfer.js — waveform rendering

License

This repo is MIT. VoxCPM2 weights are Apache-2.0 (commercial use OK) — see the model card for details.