Qwen3.6-35B-A3B Two-Blackhole Runtime Bundle

This is a Tenstorrent-facing handoff bundle for the current Stage 2 Qwen3.6-35B-A3B two-Blackhole path.

The goal is reproducibility: a recipient with two Blackhole p150a cards and the same checked-out workspace should be able to verify the environment, run the validated demo, and inspect the exact remaining host dependencies.

Status

Supported now:

• two Blackhole p150a cards;
• Qwen3.6-35B-A3B GGUF/tokenizer source;
• semantic N8/N16 fixed-region hybrid decode;
• greedy/argmax only;
• single-request-at-a-time server entrypoint;
• no CPU transformer-compute fallback in the validated semantic path.

Not supported yet:

• generic Hugging Face drop-in loading;
• conversion-free bring-up without the prebuilt artifacts listed below;
• concurrent HTTP queueing;
• production KV/session reuse;
• continuous batching;
• fully device-owned generated-token layer dispatch.

Expected Results

Validated CLI artifacts:

• N8 semantic hybrid: 10.765833137574859 tok/s;
• N16 semantic hybrid: 10.531379333280588 tok/s;
• both report semantic_full_decode_evidence=true.

Validated server artifact:

• one Stage 2 HTTP request completed with semantic evidence;
• HTTP path measured generated_model_tok_s=2.7852093390448216;
• concurrent HTTP queue smoke is blocked and is not a release check.

Required Artifacts

These paths are expected relative to the repository root:

• llm/cache/qwen36_resident_cache/
• llm/projects/qwen36-35b-a3b-fp8-two-blackhole/results/tt_lang_spike/same_source_compressed_owner_bank_cache_v0/
• llm/projects/qwen36-35b-a3b-fp8-two-blackhole/tt_lang_spike/.venv-tt-lang-py312/bin/python

The prebuilt resident cache tree is about 54 GB locally and includes the 22.1 GB GGUF/tokenizer source plus the materialized tensorbins. The compressed owner-bank sidecar cache tree is about 17 GB locally. These are intentionally referenced, not duplicated in this small wrapper directory.

Quickstart

From this package directory:

./setup_and_run.sh

If large artifacts are delivered separately:

ARTIFACT_SOURCE=/path/to/qwen36-two-blackhole-artifacts ./setup_and_run.sh

ARTIFACT_SOURCE can contain either repo-relative paths or the two top-level directories qwen36_resident_cache/ and same_source_compressed_owner_bank_cache_v0/. The default install mode uses symlinks; set INSTALL_MODE=copy only when the recipient explicitly wants a physical copy.

To start the single-request server:

./setup_and_run.sh server

Then call:

curl -s http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"Paris is the capital of"}],"max_tokens":8,"do_sample":false}'

Acceptance Test

For a recipient, the recommended first check is:

./setup_and_run.sh

This runs the environment check and the validated semantic-n8-hybrid-full-layer demo entrypoint.

Files

• setup_and_run.sh: one-command artifact install, environment check, and validated demo run.
• DOWNLOAD_AND_RUN.md: minimal recipient instructions.
• manifest.json: machine-readable bundle manifest and known metrics.
• TT_HANDOFF.md: concise handoff note for Tenstorrent reviewers.
• TT_EMAIL_UPDATE_DRAFT.md: concise update email draft.
• ARTIFACTS.md: required artifacts and copy policy.
• GITHUB_HF_RELEASE.md: recommended GitHub/Hugging Face release split.
• scripts/install_artifacts.sh: links or copies large artifacts from ARTIFACT_SOURCE.
• scripts/check_environment.sh: checks device visibility and required files.
• scripts/run_demo.sh: runs the validated CLI demo.
• scripts/run_server.sh: starts the Stage 2 single-request server.
• scripts/run_acceptance.sh: environment check plus demo run.
• scripts/print_bundle_report.sh: prints a support report for debugging.

Known Stop Rule

Do not use concurrent HTTP queueing as the acceptance test. The queue probe completed one semantic Stage 2 request, then the second concurrent request and live health stopped responding. The cards recovered after tt-smi -r all and tt-smi -ls. Keep serving single-request-at-a-time for this handoff.