A small, reproducible deployment recipe for serving
drowzeys/keys-DeepSeekV4-Flash-GA-0731-Dspark-Abliterated-32-32
across two NVIDIA DGX Spark nodes with tensor parallelism, DSpark speculative
decoding, and an OpenAI-compatible vLLM endpoint.
This repository contains only the promoted operational recipe. It intentionally does not contain optimization controllers, search prompts, experiment-selection logic, raw sessions, rejected candidates, or private benchmark logs.
Warning
The referenced checkpoint is an abliterated model with reduced refusal behavior. It may produce unsafe, illegal, inaccurate, or offensive output. Operators are responsible for access control, policy enforcement, monitoring, and compliance with all applicable terms and laws.
- Model revision:
6c47916f85e52b5e712223ca8f93952f90255714 - Runtime image:
ghcr.io/anemll/dspark-vllm-gx10:0.1.1 - Runtime digest:
sha256:a83948492cf13df455170fb42885f5ef4db54fefe0feff0f841ecbff464ac9d8 - Topology: two DGX Spark nodes, tensor parallel size 2, worker rank starts first
- KV cache:
nvfp4_ds_mla, 256-token blocks - Default profile: 1,048,576-token ceiling, 8 active sequences, 8,192 batched
tokens, 0.80 GPU memory utilization, DSpark
k=3
The image digest and model commit make the software inputs repeatable. Hardware, firmware, driver, kernel, networking, and ambient load still affect results.
On the head node:
git clone <THIS_REPOSITORY_URL> deepseek-dspark-optimized
cd deepseek-dspark-optimized
cp .env.example .envEdit .env with the worker SSH target, remote directory, fabric addresses,
RDMA devices, interfaces, and cache paths. See Two-node setup
for discovery commands and firewall guidance.
Pull the exact image on both nodes:
docker pull ghcr.io/anemll/dspark-vllm-gx10:0.1.1@sha256:a83948492cf13df455170fb42885f5ef4db54fefe0feff0f841ecbff464ac9d8Download and verify the exact model snapshot on each node. On the head:
./scripts/prepare-model.shOn the worker, clone/copy this repository, copy the same .env, then run:
THIS_NODE_HF_CACHE=/path/to/worker/cache ./scripts/prepare-model.shValidate without starting containers, then launch from the head:
./scripts/validate.sh
./scripts/launch.sh
./scripts/health-check.shThe launcher copies the public compose file, profile files, node helper, and
your untracked .env to WORKER_DIR, validates both ranks, starts the worker,
then starts the head. It does not download weights automatically.
General mode is selected by default:
./scripts/select-profile.sh generalFor coding-heavy traffic:
./scripts/select-profile.sh codingChanging the selected profile does not restart anything. Use the controlled stop/launch sequence in Operations and rollback to apply it.
./scripts/status.sh
./scripts/health-check.sh
TAIL=200 ./scripts/logs.sh
./scripts/stop.sh- Prerequisites and two-node setup
- Profiles and configuration
- Operations, rollback, and troubleshooting
- Benchmark summary and limitations
- 24-hour soak and quality certification
- Security guidance
- Credits and upstream attribution
- Third-party notices
- Publication validation record
The repository-local scripts and documentation are released under the MIT License. Model weights, the container image, and runtime dependencies are separate works governed by their respective upstream terms.
This repository does not contain model weights. The upstream repository is
gated, labels its license as deepseek, and requires acceptance of a Responsible
Use Agreement. Downloaded weights remain subject to those conditions and any
other upstream terms. The container image,
vLLM, CUDA, NCCL, FlashInfer, and other runtime components have their own terms.
Review all of them before use or redistribution.
Before changing a model revision, runtime image, or operational profile, repeat the validation and sensitive-data checks in the publication record. The initial release explicitly waived a continuous 24-hour soak and must not be described as 24-hour soak-certified.