Repository navigation
feat(domino): register training and wire Plan, trainer and checkpoint #48
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| { | ||
| "architectures": ["DominoDraftModel"], | ||
| "model_type": "domino", | ||
| "projector_type": "domino", | ||
| "hidden_size": 4096, | ||
| "intermediate_size": 12288, | ||
| "num_hidden_layers": 5, | ||
| "num_attention_heads": 32, | ||
| "num_key_value_heads": 8, | ||
| "vocab_size": 151936, | ||
| "rms_norm_eps": 1e-6, | ||
| "max_position_embeddings": 40960, | ||
| "rope_theta": 1000000.0, | ||
| "num_target_layers": 5, | ||
| "target_hidden_size": 4096, | ||
| "target_num_hidden_layers": 36, | ||
| "target_layer_ids": [1, 9, 17, 25, 33], | ||
| "mask_token_id": 151669, | ||
| "block_size": 16, | ||
| "emb_dim": 256, | ||
| "gru_hidden_dim": 1024, | ||
| "shift_label": true, | ||
| "pure_draft_prefix_len": 1, | ||
| "tie_word_embeddings": false | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,60 @@ | ||
| # Copyright 2026 Ant Group Co., Ltd. | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| """Domino trainer — DFlash backbone plus a GRU-conditioned correction head.""" | ||
|
|
||
| from argparse import Namespace | ||
|
|
||
| from aspec.models.domino import DominoModel | ||
| from aspec.models.draft.domino import DominoConfig, DominoDraftModel | ||
| from aspec.training.dflash_trainer import DFlashTrainer | ||
|
|
||
|
|
||
| class DominoTrainer(DFlashTrainer): | ||
| """Domino-specific trainer; reuses the DFlash optimizer/checkpoint path.""" | ||
|
|
||
| _draft_config_class = DominoConfig | ||
|
|
||
| def __init__(self, args: Namespace): | ||
| super().__init__(args) | ||
| self.lambda_base = getattr(args, "domino_lambda_base", 0.3) | ||
|
|
||
| def _build_draft_model(self, config): | ||
| if config.block_size != self.block_size: | ||
| raise ValueError( | ||
| "training.dflash_block_size must match domino_config.block_size " | ||
| f"({self.block_size} != {config.block_size})" | ||
| ) | ||
| if config.num_target_layers != self.num_target_layers: | ||
| raise ValueError( | ||
| "training.dflash_num_target_layers must match the number of " | ||
| f"domino_config.target_layer_ids ({self.num_target_layers} != " | ||
| f"{config.num_target_layers})" | ||
| ) | ||
| return DominoDraftModel(config) | ||
|
|
||
| def _build_training_wrapper(self, draft_model): | ||
| wrapper = DominoModel( | ||
| draft_model=draft_model, | ||
| block_size=self.block_size, | ||
| num_anchors=self.num_anchors, | ||
| loss_objective=self.loss_objective, | ||
| dpace_alpha=self.dpace_alpha, | ||
| loss_decay_gamma=self.loss_decay_gamma, | ||
| ce_loss_alpha=self.ce_loss_alpha, | ||
| l1_loss_alpha=self.l1_loss_alpha, | ||
| lambda_base=self.lambda_base, | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [P2] Report the mixed objective used for backward. |
||
| deterministic_anchor_sampling=getattr( | ||
| self.args, "deterministic_anchor_sampling", False | ||
| ), | ||
| anchor_seed=getattr(self.args, "seed", 0), | ||
| anchor_sampling_scope=getattr(self.args, "anchor_sampling_scope", "row"), | ||
| attention_backend=getattr(self.args, "attention_backend", "sdpa"), | ||
| draft_context_parallel_size=getattr( | ||
| self.args, "draft_context_parallel_size", 1 | ||
| ), | ||
| ) | ||
| # Shifted labels supervise block position 0; unshifted labels skip the | ||
| # anchor slot. Metric aggregators must drop the same prefix. | ||
| self._anchor_slot_offset = wrapper.prediction_start | ||
| return wrapper | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,93 @@ | ||
| # Supplementary configuration example; see the support matrix for supported workflows. | ||
| # Single-node Domino on a frozen Qwen3-8B Target: one Target GPU plus one Draft GPU, | ||
| # 4K, CP1, no packing, SDPA, Store/TCP transport (no NVLink). The short window | ||
| # validates the execution path, not convergence, acceptance or decoding quality. | ||
|
|
||
| model: | ||
| target_model_path: ${oc.env:ASPEC_TARGET_MODEL} | ||
| native_context_length: 32768 | ||
| trust_remote_code: false | ||
| draft_model_config: aspec/config/domino_qwen3_8b_draft_config.json | ||
|
|
||
| dataset: | ||
| train_data_path: ../../examples/data/sample_conversations.jsonl | ||
| eval_data_path: null | ||
| chat_template: qwen3-instruct | ||
| min_loss_tokens: 32 | ||
| shuffle_dataset: false | ||
|
|
||
| training: | ||
| attention_backend: sdpa | ||
| draft_context_parallel_size: 1 | ||
| micro_batch_size: 1 | ||
| draft_accumulation_steps: 4 | ||
| learning_rate: 6e-4 | ||
| min_lr: 6e-5 | ||
| weight_decay: 0.0 | ||
| max_concurrent_batches: 1 | ||
| streaming_dispatch_with_training: true | ||
| pipeline_dispatch_with_training: true | ||
| max_grad_norm: 1.0 | ||
| max_seq_length: 4096 | ||
| num_epochs: 100 | ||
| num_train_steps: 8 | ||
| lr_total_steps: 8 | ||
| seed: 42 | ||
| training_num_gpus_per_node: 2 | ||
| training_num_nodes: 1 | ||
| placement_strategy: inference_first | ||
| fsdp_strategy: REPLICATE | ||
| fsdp_reduce_dtype: bfloat16 | ||
| prefetch_depth: 1 | ||
| save_interval: 0 | ||
| save_per_epoch: false | ||
| save_final_checkpoint: true | ||
| warmup_ratio: 0.04 | ||
| dflash_packing: false | ||
| dflash_block_size: 16 | ||
| dflash_num_target_layers: 5 | ||
| dflash_num_anchors: 128 | ||
| dflash_loss_objective: decay | ||
| dflash_ce_loss_alpha: 1.0 | ||
| dflash_l1_loss_alpha: 0.0 | ||
| domino_lambda_base: 0.3 | ||
| gradient_checkpointing: true | ||
|
|
||
| inference: | ||
| inference_engine_type: sgl | ||
| store_last_hidden_states: true | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [P2] Disable the unused final Target hidden state in the TCP example. Domino uses token-label CE with |
||
| target_prefill_memory_optimized: false | ||
| inference_num_gpus: 1 | ||
| inference_num_gpus_per_engine: 1 | ||
| inference_num_gpus_per_node: 2 | ||
| inference_fetch_batch: 8 | ||
| inference_buffer_threshold: 2 | ||
| max_sample_pool_size: 12 | ||
| inference_batch_size: 2 | ||
| sglang: | ||
| tp_size: 1 | ||
| mem_fraction_static: 0.5 | ||
|
|
||
| mooncake: | ||
| master_server_address: null | ||
| metadata_server: null | ||
| data_plane: store | ||
| protocol: tcp | ||
| enable_gpu_direct: false | ||
| allow_host_fallback: false | ||
| global_segment_size: 32GB | ||
| local_buffer_size: 8GB | ||
| host_buffer_size: 1GB | ||
| async_put_pool_size: 1 | ||
| enable_hard_pin: true | ||
| max_seq_len: 4096 | ||
| hidden_dim: 4096 | ||
|
|
||
| logging: | ||
| report_to: none | ||
|
|
||
| debug: | ||
| enable_perf_metrics: true | ||
|
|
||
| output_dir: outputs/domino_qwen3_8b_4k_tcp | ||
| cache_dir: cache/domino_qwen3_8b_4k_tcp | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[P1] Match the declared attention backend to the masked training path. This gate only accepts
sdpa. On CUDA,DFlashModel._draft_backbonebuilds aBlockMask, which sendsDFlashAttention.forwardintocompile_friendly_flex_attention. On CPU the mask is absent, so SDPA can attend to future Target features and other draft blocks. The current CPU loss checks reuse the resulting backbone activations and can pass even with that leakage. Please align the Plan and kernel dispatch, and add a visibility regression on the supported path: with anchors fixed, changing Target features after an anchor must leave that block’s logits unchanged.