⚠️ X-Talk is in active prototyping. Interfaces and functions are subject to change. We will try to keep interfaces stable.
X-Talk is an open-source full-duplex cascaded spoken dialogue system framework featuring:
- ⚡ Low-Latency, Interruptible, Human-Like Speech Interaction
- Speech flow is optimized to support impressive low latency
- Enables natural user interruption during interaction
- Paralinguistic information (e.g. environment noise, emotion) is encoded in parallel to support in-depth understanding and empathy
- 🧪 Researcher Friendly
- New models and relevant logic can be added within one Python script, and seamlessly integrated with the default pipeline.
- 🧩 Super Lightweight
- The framework backend is pure Python; nothing to build and install beyond
pip install.
- The framework backend is pure Python; nothing to build and install beyond
- 🏭 Production Ready
- Concurrency is ensured through asynchronous backend
- Websocket-based implementation empowers deployment from web browsers to edge devices.
This demo runs on 4090 cluster with 8-bit quantized SenseVoice as speech recognizer, IndexTTS 1.5 as speech generator, and 4-bit quantized Qwen3-30B-A3B as language model. Though at the cost of intelligence due to a relatively small language model, it demonstrates low latency.
tour-guide-en.mp4 |
tour-guide-zh.mp4 |
twenty-questions-en.mp4 |
word-chain-game-zh.mp4 |
web-search-en.mp4 |
web-search-zh.mp4 |
noisy-scene-en.mp4 |
noisy-scene-zh.mp4 |
multi-speaker-en.mp4 |
multi-speaker-zh.mp4 |
The tour guiding demos are conducted with Qwen3-Next-80B-A3B-Instruct as language model, and the other eight demos are aligned with the online demo setting. Larger language models are more intelligent at the cost of latency.
pip install git+https://github.com/xcc-zach/xtalk.git@mainWe will use APIs from AliCloud to demonstrate the basic capability of X-Talk.
First, install dependencies for AliCloud and server script:
pip install "xtalk[ali,example] @ git+https://github.com/xcc-zach/xtalk.git@main"Then, obtain an API key from AliCloud Bailian Platform. We will be using free-tier service (currently) from AliCloud.
Online service may be unstable and of high latency. We recommend using locally deployed models for better user experience. See server config tutorial and local deployment recipe for details.
After that, create a JSON config specifying the models to use, and fill in <API_KEY> with the key you obtained:
{
"asr": {
"type": "Qwen3ASRFlashRealtime",
"params": {
"api_key": "<API_KEY>"
}
},
"llm_agent": {
"type": "DefaultAgent",
"params": {
"model": {
"api_key": "<API_KEY>",
"model": "qwen-plus-2025-12-01",
"base_url": "https://dashscope.aliyuncs.com/compatible-mode/v1"
}
}
},
"tts": {
"type": "CosyVoice",
"params": {
"api_key": "<API_KEY>"
}
}
}The next step is to compose the startup script. Since we also need to link frontend webpage and scripts to get the demo working, the startup script is ready at examples/sample_app/configurable_server.py. We simply need to start the server with the config file (fill in <PATH_TO_CONFIG>.json with the path to the config file we just created) and a custom port:
git clone https://github.com/xcc-zach/xtalk.git
cd xtalk
python examples/sample_app/configurable_server.py --port 7635 --config <PATH_TO_CONFIG>.jsonFinally, our demo is ready at http://localhost:7635. View it in the browser!
Docs here
We express sincere gratitude for:
- Langchain as backbone of LLM agents
- vllm for deployment of most models
- All model providers mentioned in Supported Models
All of you provide the solid foundation of X-Talk!
This project is licensed under the Apache License 2.0, if you do not install optional dependencies. Some optional dependencies may be under incompatible licenses.