enable pcp for dsa models#1514
Draft
whx-sjtu wants to merge 18 commits into
Draft
Conversation
Signed-off-by: zejunchen-zejun <zejun.chen@amd.com>
Signed-off-by: zejunchen-zejun <zejun.chen@amd.com>
Signed-off-by: zejunchen-zejun <zejun.chen@amd.com>
Signed-off-by: zejunchen-zejun <zejun.chen@amd.com>
Signed-off-by: zejunchen-zejun <zejun.chen@amd.com>
GLM-5.2 (glm_moe_dsa) was hard-excluded from the fused indexer path added for DeepSeek-V3.2 in #788, forcing the indexer's rope + fp8-quant + kv-cache write to run as 5-6 separate ops per layer per token. GLM's indexer is structurally identical to V3.2 (index_head_dim=128, qk_rope_head_dim=64, per_1x128 fp8 quant, always-neox indexer rope), so the fused indexer_qk_rope_quant_and_cache kernel is math-equivalent to the per-op path. Notably the recent GLM acc fix (da360a2) fixed a bug that only exists in the per-op path (dropping q_nope on recombine); the fused kernel never had it. - Allow glm_moe_dsa through _supports_fused_indexer_kernel_config, gated by a new env ATOM_ENABLE_GLM_FUSED_INDEXER (default on) for easy A/B + rollback. - Keep the wk+weights_proj GEMM merge OFF for GLM (decoupled in _can_fuse_indexer_wk_weights_proj): that fusion requires merging two checkpoint tensors and GLM names weights_proj "indexers_proj" without a pre-mergeable layout. The dominant win (rope+quant+cache) needs no merge. DeepSeek-V3.2 behavior is unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Remove the guard that kept the indexer wk+weights_proj GEMM-merge off for glm_moe_dsa. The guard was overly conservative: it assumed GLM checkpoints ship the projection under a non-standard "indexers_proj" name that couldn't feed the packed_modules_mapping merge. In fact "indexers_proj" only appears in the HF quant config's modules_to_not_convert (remapped via quant_exclude_name_mapping); the actual checkpoint tensors use the standard indexer.wk (fp8 block-scale) + indexer.weights_proj (bf16) paths — exactly the layout IndexerWkWeightsProjLinear's fp8-wk load and the merge expect. GLM now takes the same path as DeepSeek-V3.2: wk + weights_proj collapse into a single BF16 GEMM (IndexerWkWeightsProjLinear), on top of the already-enabled qk-rope+quant+cache fusion. Still gated by ATOM_ENABLE_GLM_FUSED_INDEXER for rollback. DeepSeek-V3.2 behavior unchanged. Verify on GPU: weights load without missing indexer.wk / indexer.weights_proj, accuracy unchanged, and the two small indexer GEMMs collapse to one in the decode breakdown. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: zejunchen-zejun <zejun.chen@amd.com>
Signed-off-by: zejunchen-zejun <zejun.chen@amd.com>
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com>
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com>
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com>
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com>
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com>
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com>
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com>
yitingw1
reviewed
Jul 8, 2026
| ): | ||
| raise ValueError( | ||
| "prefill_context_parallel_size > 1 (-pcp) is currently only " | ||
| "supported for sparse-MLA / DSA models (DeepSeek-V3.2, GLM-5.2 — " |
Contributor
There was a problem hiding this comment.
DeepSeekV4 is also supported.
yitingw1
reviewed
Jul 8, 2026
| ) | ||
| # PCP v1 requires prefix caching OFF: mixing a full cached KV prefix with | ||
| # 1/pcp new tokens in the indexer gather is not yet handled. | ||
| if config.prefill_context_parallel_size > 1 and config.enable_prefix_caching: |
Contributor
There was a problem hiding this comment.
PCP for DeepseekV4 is compatible with prefix cache and chunked prefill. It should be suitable for DSA, too.
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Enable PCP for DSA models
Test results