Skip to content

Commit 8a447f0

Browse files
juntaojuntao
authored
Merge pull request #7 from second-state/add-rust-server
Add Rust-based OpenAI-compatible audio server
2 parents 74600b3 + 197e1f0 commit 8a447f0

File tree

18 files changed

+4995
-3
lines changed

18 files changed

+4995
-3
lines changed

.github/workflows/ci.yml renamed to .github/workflows/ci-python.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
name: CI
1+
name: CI (Python)
22

33
on:
44
push:

‎.github/workflows/ci-rust.yml‎

Lines changed: 484 additions & 0 deletions
Large diffs are not rendered by default.

‎.github/workflows/release-rust.yml‎

Lines changed: 147 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,147 @@
1+
name: Release (Rust)
2+
3+
on:
4+
push:
5+
tags:
6+
- 'v*'
7+
8+
permissions:
9+
contents: write
10+
11+
jobs:
12+
build-linux:
13+
runs-on: ubuntu-latest
14+
timeout-minutes: 60
15+
16+
env:
17+
LIBTORCH_VERSION: "2.7.0"
18+
LIBTORCH_BYPASS_VERSION_CHECK: "1"
19+
20+
steps:
21+
- uses: actions/checkout@v4
22+
23+
- name: Install Rust toolchain
24+
uses: dtolnay/rust-toolchain@stable
25+
26+
- name: Install system dependencies
27+
run: |
28+
sudo apt-get update && sudo apt-get install -y ffmpeg cmake \
29+
libavformat-dev libavcodec-dev libavutil-dev libavfilter-dev \
30+
libavdevice-dev libswscale-dev libswresample-dev libclang-dev pkg-config
31+
32+
- name: Cache libtorch
33+
id: cache-libtorch
34+
uses: actions/cache@v4
35+
with:
36+
path: libtorch
37+
key: libtorch-cpu-${{ env.LIBTORCH_VERSION }}
38+
39+
- name: Download libtorch
40+
if: steps.cache-libtorch.outputs.cache-hit != 'true'
41+
run: |
42+
wget -q "https://download.pytorch.org/libtorch/cpu/libtorch-cxx11-abi-shared-with-deps-${LIBTORCH_VERSION}%2Bcpu.zip" -O libtorch.zip
43+
unzip -q libtorch.zip
44+
rm libtorch.zip
45+
46+
- name: Set libtorch environment
47+
run: |
48+
echo "LIBTORCH=$(pwd)/libtorch" >> $GITHUB_ENV
49+
echo "LD_LIBRARY_PATH=$(pwd)/libtorch/lib:$LD_LIBRARY_PATH" >> $GITHUB_ENV
50+
51+
- name: Cache cargo registry & build
52+
uses: actions/cache@v4
53+
with:
54+
path: |
55+
~/.cargo/registry
56+
~/.cargo/git
57+
rust/target
58+
key: cargo-release-linux-${{ hashFiles('rust/Cargo.toml') }}
59+
restore-keys: cargo-release-linux-
60+
61+
- name: Build
62+
working-directory: rust
63+
run: cargo build --release
64+
65+
- name: Package archive
66+
run: |
67+
mkdir -p staging/qwen3-audio-api-linux-x86_64
68+
cp rust/target/release/qwen3-audio-api staging/qwen3-audio-api-linux-x86_64/
69+
cp -r libtorch staging/qwen3-audio-api-linux-x86_64/
70+
tar -czf qwen3-audio-api-linux-x86_64.tar.gz -C staging qwen3-audio-api-linux-x86_64
71+
72+
- name: Upload artifact
73+
uses: actions/upload-artifact@v4
74+
with:
75+
name: linux-x86_64
76+
path: qwen3-audio-api-linux-x86_64.tar.gz
77+
78+
build-macos:
79+
runs-on: macos-14
80+
timeout-minutes: 60
81+
82+
steps:
83+
- uses: actions/checkout@v4
84+
85+
- name: Install Rust toolchain
86+
uses: dtolnay/rust-toolchain@stable
87+
88+
- name: Install system dependencies
89+
run: brew install ffmpeg cmake
90+
91+
- name: Cache cargo registry & build
92+
uses: actions/cache@v4
93+
with:
94+
path: |
95+
~/.cargo/registry
96+
~/.cargo/git
97+
rust/target
98+
key: cargo-release-macos-arm64-${{ hashFiles('rust/Cargo.toml') }}
99+
restore-keys: cargo-release-macos-arm64-
100+
101+
- name: Build with MLX backend
102+
working-directory: rust
103+
run: cargo build --release --no-default-features --features mlx
104+
105+
- name: Package archive
106+
run: |
107+
METALLIB=$(find rust/target/release/build -path '*/qwen3_tts-*/out/lib/mlx.metallib' | head -1)
108+
if [ -z "$METALLIB" ]; then
109+
echo "ERROR: mlx.metallib not found in build artifacts"
110+
exit 1
111+
fi
112+
echo "Found metallib at: $METALLIB ($(stat -f%z "$METALLIB") bytes)"
113+
mkdir -p staging/qwen3-audio-api-macos-arm64
114+
cp rust/target/release/qwen3-audio-api staging/qwen3-audio-api-macos-arm64/
115+
cp "$METALLIB" staging/qwen3-audio-api-macos-arm64/
116+
tar -czf qwen3-audio-api-macos-arm64.tar.gz -C staging qwen3-audio-api-macos-arm64
117+
118+
- name: Upload artifact
119+
uses: actions/upload-artifact@v4
120+
with:
121+
name: macos-arm64
122+
path: qwen3-audio-api-macos-arm64.tar.gz
123+
124+
release:
125+
needs: [build-linux, build-macos]
126+
runs-on: ubuntu-latest
127+
permissions:
128+
contents: write
129+
130+
steps:
131+
- name: Download Linux artifact
132+
uses: actions/download-artifact@v4
133+
with:
134+
name: linux-x86_64
135+
136+
- name: Download macOS artifact
137+
uses: actions/download-artifact@v4
138+
with:
139+
name: macos-arm64
140+
141+
- name: Create GitHub Release
142+
uses: softprops/action-gh-release@v2
143+
with:
144+
generate_release_notes: true
145+
files: |
146+
qwen3-audio-api-linux-x86_64.tar.gz
147+
qwen3-audio-api-macos-arm64.tar.gz

‎README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,11 +10,11 @@ OpenAI-compatible API servers for [Qwen3-TTS](https://github.com/QwenLM/Qwen3-TT
1010
| Language | Directory | Status |
1111
|----------|-----------|--------|
1212
| Python | [python/](python/) | Available |
13-
| Rust | `rust/` (planned) | Coming soon |
13+
| Rust | [rust/](rust/) | Available |
1414

1515
The **Python** implementation is a FastAPI server built on the `qwen-tts` and `qwen-asr` Python packages. See [python/README.md](python/README.md) for setup, Docker images, API reference, and usage examples.
1616

17-
The **Rust** implementation will be built on the [qwen3_tts](https://crates.io/crates/qwen3_tts) Rust crate. Stay tuned.
17+
The **Rust** implementation is a high-performance axum/tokio server built on the [qwen3_tts](https://github.com/second-state/qwen3_tts_rs) and [qwen3_asr](https://github.com/second-state/qwen3_asr_rs) Rust crates, with libtorch (Linux) and MLX (macOS Apple Silicon) backends. See [rust/README.md](rust/README.md) for setup, pre-built binaries, API reference, and usage examples.
1818

1919
## Features
2020

0 commit comments

Comments
 (0)