ttt: add EPLB and Frontier-CS self-distillation examples#68
Open
Yuchen-95 wants to merge 5 commits intoHuman-Agent-Society:feat/self_distillfrom
Open
ttt: add EPLB and Frontier-CS self-distillation examples#68Yuchen-95 wants to merge 5 commits intoHuman-Agent-Society:feat/self_distillfrom
Yuchen-95 wants to merge 5 commits intoHuman-Agent-Society:feat/self_distillfrom
Conversation
Add two new TTT example tasks for self-distillation training, plus a generator script to scale Frontier-CS to all 43 CPU-only problems. New examples: - ttt/examples/eplb/: Expert Parallelism Load Balancer (MoE optimization) - ttt/examples/frontier_cs/: Frontier-CS cant_be_late scheduling - ttt/examples/frontier_cs_tasks/: 43 generated CPU-only Frontier-CS problems Setup (EPLB): pip install -e . && pip install -e ttt/slime wget https://huggingface.co/datasets/abmfy/eplb-openevolve/resolve/main/expert-load.json cp expert-load.json ttt/examples/eplb/eval/expert-load.json CORAL_TASK_YAML=ttt/examples/eplb/task.yaml ./ttt/run_coral_distill.sh Setup (Frontier-CS): pip install -e . && pip install -e ttt/slime pip install git+https://github.com/FrontierCS/Frontier-CS.git # Docker required for evaluation CORAL_TASK_YAML=ttt/examples/frontier_cs/task.yaml ./ttt/run_coral_distill.sh Generate all 43 CPU-only Frontier-CS tasks: python ttt/examples/frontier_cs/generate_tasks.py CORAL_TASK_YAML=ttt/examples/frontier_cs_tasks/cloudcast/task.yaml ./ttt/run_coral_distill.sh Both examples use opencode runtime with restricted tools (no LSP, no codesearch, no skills) to keep 4B models focused on coding. Also adds frontier-cs as an optional dependency in pyproject.toml: pip install -e '.[frontier-cs]'
|
@Yuchen-95 is attempting to deploy a commit to the yhy-4770's projects Team on Vercel. A member of the Team first needs to authorize it. |
The pip-installed frontier_cs package can't auto-detect its base directory since the research/ folder isn't included in the wheel. Pass base_dir explicitly pointing to the cloned Frontier-CS repo (~/Frontier-CS by default, configurable via FRONTIER_CS_BASE_DIR env var).
The generic Solution stub didn't have the correct Strategy inheritance or imports (sky_spot.strategies.strategy.Strategy, ClusterType), causing NameError at evaluation time. Replace with the actual initial_greedy.py from the Frontier-CS repo.
The KL entropy computation materializes [batch, seq, vocab] tensors that OOM on single A100-40G with 4B+ models. KL loss is now disabled by default and can be enabled via USE_KL_LOSS=1 for multi-GPU setups.
The _compute_log_prob() path was always computing entropy (full vocab softmax), while _train_step() correctly gated it. This caused OOM on A100-40G with 4B models because the [seq_len, vocab_size] tensor couldn't fit alongside model+optimizer states.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add two new TTT example tasks for self-distillation training, plus a generator script to scale Frontier-CS to all 43 CPU-only problems.
New examples:
Setup (EPLB):
pip install -e . && pip install -e ttt/slime wget https://huggingface.co/datasets/abmfy/eplb-openevolve/resolve/main/expert-load.json cp expert-load.json ttt/examples/eplb/eval/expert-load.json CORAL_TASK_YAML=ttt/examples/eplb/task.yaml ./ttt/run_coral_distill.sh
Setup (Frontier-CS):
pip install -e . && pip install -e ttt/slime pip install git+https://github.com/FrontierCS/Frontier-CS.git # Docker required for evaluation CORAL_TASK_YAML=ttt/examples/frontier_cs/task.yaml ./ttt/run_coral_distill.sh
Generate all 43 CPU-only Frontier-CS tasks:
python ttt/examples/frontier_cs/generate_tasks.py CORAL_TASK_YAML=ttt/examples/frontier_cs_tasks/cloudcast/task.yaml ./ttt/run_coral_distill.sh
Both examples use opencode runtime with restricted tools (no LSP, no codesearch, no skills) to keep 4B models focused on coding.
Also adds frontier-cs as an optional dependency in pyproject.toml:
pip install -e '.[frontier-cs]'