Add_word_level_timestamps() always groups subword tokens with
split_tokens_on_spaces(), which depends on leading spaces to find word boundaries
This does not work for no-space (Scriptio continua) languages (Chinese, Japanese, Thai, Lao, Burmese, Cantonese) where decoded text has no spaces.
I made a fix in 2026.2 branch of my own fork which seems to work (afaik)
cea5a80
I could potentially make a pull request against main, but the changes are pretty small and only in a single file, so it might be easier for maintainers to just fix directly.
Add_word_level_timestamps() always groups subword tokens with
split_tokens_on_spaces(), which depends on leading spaces to find word boundaries
This does not work for no-space (Scriptio continua) languages (Chinese, Japanese, Thai, Lao, Burmese, Cantonese) where decoded text has no spaces.
I made a fix in 2026.2 branch of my own fork which seems to work (afaik)
cea5a80
I could potentially make a pull request against main, but the changes are pretty small and only in a single file, so it might be easier for maintainers to just fix directly.