Skip to content

About

Optimized two-node DGX Spark deployment recipe for DeepSeek V4 Flash with vLLM, DSpark, and NVFP4 KV cache

Topics

Resources

Security policy

Stars

11 stars

Watchers

0 watching

Forks

Repository files navigation

DeepSeek V4 Flash 0731 on two DGX Sparks

A small, reproducible deployment recipe for serving drowzeys/keys-DeepSeekV4-Flash-GA-0731-Dspark-Abliterated-32-32 across two NVIDIA DGX Spark nodes with tensor parallelism, DSpark speculative decoding, and an OpenAI-compatible vLLM endpoint.

This repository contains only the promoted operational recipe. It intentionally does not contain optimization controllers, search prompts, experiment-selection logic, raw sessions, rejected candidates, or private benchmark logs.

Warning

The referenced checkpoint is an abliterated model with reduced refusal behavior. It may produce unsafe, illegal, inaccurate, or offensive output. Operators are responsible for access control, policy enforcement, monitoring, and compliance with all applicable terms and laws.

What is pinned

  • Model revision: 6c47916f85e52b5e712223ca8f93952f90255714
  • Runtime image: ghcr.io/anemll/dspark-vllm-gx10:0.1.1
  • Runtime digest: sha256:a83948492cf13df455170fb42885f5ef4db54fefe0feff0f841ecbff464ac9d8
  • Topology: two DGX Spark nodes, tensor parallel size 2, worker rank starts first
  • KV cache: nvfp4_ds_mla, 256-token blocks
  • Default profile: 1,048,576-token ceiling, 8 active sequences, 8,192 batched tokens, 0.80 GPU memory utilization, DSpark k=3

The image digest and model commit make the software inputs repeatable. Hardware, firmware, driver, kernel, networking, and ambient load still affect results.

Quick start

On the head node:

git clone <THIS_REPOSITORY_URL> deepseek-dspark-optimized
cd deepseek-dspark-optimized
cp .env.example .env

Edit .env with the worker SSH target, remote directory, fabric addresses, RDMA devices, interfaces, and cache paths. See Two-node setup for discovery commands and firewall guidance.

Pull the exact image on both nodes:

docker pull ghcr.io/anemll/dspark-vllm-gx10:0.1.1@sha256:a83948492cf13df455170fb42885f5ef4db54fefe0feff0f841ecbff464ac9d8

Download and verify the exact model snapshot on each node. On the head:

./scripts/prepare-model.sh

On the worker, clone/copy this repository, copy the same .env, then run:

THIS_NODE_HF_CACHE=/path/to/worker/cache ./scripts/prepare-model.sh

Validate without starting containers, then launch from the head:

./scripts/validate.sh
./scripts/launch.sh
./scripts/health-check.sh

The launcher copies the public compose file, profile files, node helper, and your untracked .env to WORKER_DIR, validates both ranks, starts the worker, then starts the head. It does not download weights automatically.

Profiles

General mode is selected by default:

./scripts/select-profile.sh general

For coding-heavy traffic:

./scripts/select-profile.sh coding

Changing the selected profile does not restart anything. Use the controlled stop/launch sequence in Operations and rollback to apply it.

Routine commands

./scripts/status.sh
./scripts/health-check.sh
TAIL=200 ./scripts/logs.sh
./scripts/stop.sh

Documentation

License

The repository-local scripts and documentation are released under the MIT License. Model weights, the container image, and runtime dependencies are separate works governed by their respective upstream terms.

Model weights and licenses

This repository does not contain model weights. The upstream repository is gated, labels its license as deepseek, and requires acceptance of a Responsible Use Agreement. Downloaded weights remain subject to those conditions and any other upstream terms. The container image, vLLM, CUDA, NCCL, FlashInfer, and other runtime components have their own terms. Review all of them before use or redistribution.

Before changing a model revision, runtime image, or operational profile, repeat the validation and sensitive-data checks in the publication record. The initial release explicitly waived a continuous 24-hour soak and must not be described as 24-hour soak-certified.

About

Optimized two-node DGX Spark deployment recipe for DeepSeek V4 Flash with vLLM, DSpark, and NVFP4 KV cache

Topics

Resources

Security policy

Stars

11 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages