Repository navigation
Expand file tree
/
Copy pathNOTICE
More file actions
34 lines (28 loc) · 1.91 KB
/
Copy pathNOTICE
File metadata and controls
34 lines (28 loc) · 1.91 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
vLLM-Moet
Copyright 2026 the vLLM-Moet authors
This product includes software developed for serving frontier Mixture-of-Experts
models (DeepSeek-V4-Flash, GLM-5.2, Kimi-K2.7-Code) on consumer/workstation SM120
GPUs: 2-bit MoE expert quantization, an FP4 "delta" tier, tiered expert residency
(host RAM / NVMe expert stores), and hand-written SASS kernels (the `cubit`
toolchain).
Built on and incorporating:
- vLLM (https://github.com/vllm-project/vllm), Apache License 2.0,
Copyright contributors to the vLLM project. The base is the official tag v0.24.0;
`patch/vllm-moet-v0.24.0.patch` is applied on top of it. DeepSeek-V4 model support,
sparse-MLA / MTP / indexer and SM120 enablement ship in upstream vLLM and were authored
by vLLM contributors — not by this project. This project's own contribution is the
2-bit MoE + FP4 delta cache + confidence gate + expert stores + the cubit kernels,
plus small SM120 fixes to the upstream base (carried in the patch).
- DeepGEMM (https://github.com/deepseek-ai/DeepGEMM), MIT License, Copyright (c) 2025
DeepSeek. Used at runtime on SM120 — FP8 dense/einsum GEMMs, the indexer's MQA logits,
hc_prenorm, and the stock DEEPGEMM_MXFP4 MoE path the MTP drafter runs on — via the
nv-dev pin `a6b593d2`, whose SM120 backend was authored by the NVIDIA DevTech APAC team
(deepseek-ai/DeepGEMM PR #324). The Docker image builds it from source at that pin;
any prebuilt `deep_gemm` wheel distributed alongside this project is covered by that
MIT license and copyright.
- Model weights (DeepSeek-V4-Flash, GLM-5.2, Kimi-K2.7-Code checkpoints) are NOT
included or redistributed. Obtain the official checkpoints separately; the 2-bit
planes are built from them at load time.
cubit (the SASS assembler and the hand-written kernels under `kernels/`) and the 2-bit
MoE + FP4-delta + expert-store code added by `patch/vllm-moet-v0.24.0.patch` are the
original contribution of this project.