Skip to content

Commit 3e7465f

Browse files
committed
docs(voice-clone): clarify --prompt-text scope, length tuning, profile reuse (#9)
The CLI help on `--prompt-text` (in both `infer.py` and `moss_tts_nano/cli.py`) said it was "used by continuation mode" — but `model.inference` accepts `prompt_text` for voice_clone mode too, and supplying it improves cloning quality. Update the help to reflect that. Also adds a "Voice cloning details" subsection to README.md and README_zh.md that addresses the three questions from #9 directly: 1. Yes, you can pass the source audio's transcript via --prompt-text / --prompt-text-file. It works for both modes. 2. Reference audio length: no enforced limit, but ~3–10 seconds of clean single-speaker speech tends to give the best results. Acknowledges the empirical observation that very short or very long clips degrade output, with a concrete suggestion (clip ~5 s). 3. There's no separate "voice profile" cache yet — keep the model loaded in process (via `python -i infer.py`, `moss-tts-nano serve`, or a reused `MossTtsNanoRuntime`) and call inference repeatedly with the same prompt args. No behavioural change; help text + docs only. Closes #9.
1 parent 7928ec1 commit 3e7465f

4 files changed

Lines changed: 76 additions & 3 deletions

File tree

README.md

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -147,6 +147,41 @@ python infer.py \
147147

148148
This writes audio to `generated_audio/infer_output.wav` by default.
149149

150+
#### Voice cloning details
151+
152+
A few questions come up frequently (see [#9](https://github.com/OpenMOSS/MOSS-TTS-Nano/issues/9)):
153+
154+
1. **Can I pass the transcript of the reference audio?**
155+
Yes — use `--prompt-text "<transcript>"` (or `--prompt-text-file path.txt`).
156+
It is honoured by both `--mode voice_clone` (the default) and
157+
`--mode continuation`. Supplying it generally improves cloning quality
158+
because the model can align text-to-audio for the prompt clip.
159+
160+
```bash
161+
python infer.py \
162+
--prompt-audio-path assets/audio/zh_1.wav \
163+
--prompt-text "欢迎收听今日新闻播报。" \
164+
--text "今天的天气非常好。"
165+
```
166+
167+
2. **What length should the reference audio be?**
168+
We don't enforce a hard limit — the audio tokenizer accepts arbitrary
169+
lengths and the prompt is internally clipped by
170+
`--max-new-frames` / `--voice-clone-max-text-tokens`. Empirically,
171+
short clips (≈ 3–10 seconds) of *clean* speech tend to give the best
172+
results: long clips spend more of the model's prompt budget on
173+
acoustic context, and very short ones (< 2 s) often don't carry
174+
enough timbre. If you see degraded output, try clipping a clean,
175+
single-speaker passage at around 5 seconds.
176+
177+
3. **How do I cache a voice profile across multiple generations?**
178+
There's no separate "voice profile" object yet — the cleanest pattern
179+
is to keep the model loaded in process (e.g. via `python -i infer.py`,
180+
`moss-tts-nano serve`, or by reusing a `MossTtsNanoRuntime` instance
181+
in your own script) and call `model.inference(...)` repeatedly with
182+
the same `prompt_audio_path` and `prompt_text`. The audio tokenizer
183+
will re-encode the prompt each call, but the model weights stay warm.
184+
150185
### Local Web Demo with `app.py`
151186

152187
You can launch the local FastAPI demo for browser-based testing:

README_zh.md

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -142,6 +142,29 @@ python infer.py \
142142

143143
默认情况下,这会将音频写入 `generated_audio/infer_output.wav`
144144

145+
#### 语音克隆细节
146+
147+
社区里反复出现的几个问题(参见 [#9](https://github.com/OpenMOSS/MOSS-TTS-Nano/issues/9)):
148+
149+
1. **可以传入参考音频的转写文本吗?**
150+
可以 — 使用 `--prompt-text "<转写>"`(或 `--prompt-text-file path.txt`)。
151+
`--mode voice_clone`(默认)和 `--mode continuation` 都支持。提供该转写
152+
通常能提升克隆质量,因为模型可以将提示片段的文本与音频对齐。
153+
154+
2. **参考音频应该多长?**
155+
没有硬性限制 — 音频 tokenizer 接受任意长度,提示部分会受
156+
`--max-new-frames` / `--voice-clone-max-text-tokens` 内部裁剪。经验上,
157+
3–10 秒左右的*干净*单人语音效果最好:过长的片段会把模型的提示预算
158+
花在声学上下文上,过短(< 2 秒)的片段又难以承载足够音色信息。
159+
如果输出质量下降,建议截取一段约 5 秒的清晰单人语音重试。
160+
161+
3. **如何在多次生成之间缓存语音 profile?**
162+
目前没有独立的"语音 profile"对象 — 最干净的做法是让模型驻留在进程中
163+
(例如 `python -i infer.py``moss-tts-nano serve`,或在脚本中复用
164+
`MossTtsNanoRuntime` 实例),然后用相同的 `prompt_audio_path`
165+
`prompt_text` 反复调用 `model.inference(...)`。音频 tokenizer 每次都会
166+
重新编码提示,但模型权重保持加载状态。
167+
145168
### 使用 `app.py` 启动本地 Web 演示
146169

147170
您可以启动本地 FastAPI 演示进行基于浏览器的测试:

infer.py

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -53,8 +53,18 @@ def parse_args(argv: Optional[Sequence[str]] = None) -> argparse.Namespace:
5353
text_group.add_argument("--text-file", help="Path to a UTF-8 text file to synthesize.")
5454

5555
prompt_text_group = parser.add_mutually_exclusive_group(required=False)
56-
prompt_text_group.add_argument("--prompt-text", help="Reference transcript used by continuation mode.")
57-
prompt_text_group.add_argument("--prompt-text-file", help="UTF-8 reference transcript file used by continuation mode.")
56+
prompt_text_group.add_argument(
57+
"--prompt-text",
58+
help=(
59+
"Transcript of the reference audio. Used by both continuation mode and "
60+
"voice_clone mode — supplying it generally improves cloning quality "
61+
"because the model can align text-to-audio for the prompt clip."
62+
),
63+
)
64+
prompt_text_group.add_argument(
65+
"--prompt-text-file",
66+
help="UTF-8 file alternative to --prompt-text. Same behaviour for both modes.",
67+
)
5868

5969
parser.add_argument("--text-tokenizer-path", default=None, help="Override the checkpoint-bundled text tokenizer.")
6070
parser.add_argument(

moss_tts_nano/cli.py

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,12 @@ def _build_parser() -> argparse.ArgumentParser:
7171
generate_parser.add_argument(
7272
"--prompt-text",
7373
default=None,
74-
help="PyTorch backend only. Reference transcript used by continuation mode.",
74+
help=(
75+
"PyTorch backend only. Transcript of the reference audio. Used by "
76+
"both continuation and voice_clone modes, and supplying it "
77+
"generally improves cloning quality because the model can align "
78+
"text-to-audio for the prompt."
79+
),
7580
)
7681
generate_parser.add_argument(
7782
"--voice",

0 commit comments

Comments
 (0)