Goal
Add phoneme-level timing alignment so consumers can map audio samples back to individual phonemes.
What is phoneme alignment
Per-phoneme start/end sample indices extracted from the model's duration predictor output. Enables use-cases like viseme lip-sync, karaoke highlighting, and pronunciation analysis.
Upstream
Ported from piper (OHF-Voice/piper1-gpl commit eb6be6b). The alignment logic does not use espeak-ng and is covered by Apache 2.
Design
- Second ONNX output tensor: per-frame phoneme IDs
hop_length (default 256) converts frames → samples
PhonemeAlignment dataclass: phoneme + num_samples
AudioChunk extended with phoneme_alignments, phoneme_ids, phoneme_id_samples
TTSVoice.synthesize(include_alignments=True) returns populated chunks when the model supports it
phoonnx_train export-onnx --add-phoneme-alignment embeds the extra output in the exported graph
PR
#125 — feat/phoneme-alignment
Deliverables
PhonemeAlignment dataclass
AudioChunk alignment fields
synthesize / phoneme_ids_to_audio alignment path
tests/test_alignment.py (24 hermetic unit tests)
docs/alignment.md (usage guide with viseme and karaoke examples)
Acceptance criteria
Goal
Add phoneme-level timing alignment so consumers can map audio samples back to individual phonemes.
What is phoneme alignment
Per-phoneme start/end sample indices extracted from the model's duration predictor output. Enables use-cases like viseme lip-sync, karaoke highlighting, and pronunciation analysis.
Upstream
Ported from piper (OHF-Voice/piper1-gpl commit eb6be6b). The alignment logic does not use espeak-ng and is covered by Apache 2.
Design
hop_length(default 256) converts frames → samplesPhonemeAlignmentdataclass:phoneme+num_samplesAudioChunkextended withphoneme_alignments,phoneme_ids,phoneme_id_samplesTTSVoice.synthesize(include_alignments=True)returns populated chunks when the model supports itphoonnx_train export-onnx --add-phoneme-alignmentembeds the extra output in the exported graphPR
#125 —
feat/phoneme-alignmentDeliverables
PhonemeAlignmentdataclassAudioChunkalignment fieldssynthesize/phoneme_ids_to_audioalignment pathtests/test_alignment.py(24 hermetic unit tests)docs/alignment.md(usage guide with viseme and karaoke examples)Acceptance criteria
--add-phoneme-alignmentproduces valid two-output graphssynthesize(include_alignments=True)returnsPhonemeAlignmentlists when model supports itsynthesize(include_alignments=True)returnsNonefields gracefully when model does not support it