Training-Free GRPO: `env_mode: local` python executor leaks zombie threads on timeout, blocking all code execution

[shallinan1]

Setup:

• Model: Qwen3-32B on 2× A100 80GB (tensor-parallel-size 2)
• vLLM 0.15.1 with --enable-auto-tool-choice --tool-call-parser hermes
• Scripts: scripts/run_training_free_GRPO.py --config_name math_reasoning for optimize, scripts/run_eval.py --config_name math/math_practice_AIME25 for eval
• rollout_concurrency: 64 (optimize), --concurrency 128 (eval)
• Vendor commit 608615b

Issue:

When running TF-GRPO with env_mode: local, execute_python_code_async (utu/tools/local_env/python.py:209-218) uses asyncio.wait_for with loop.run_in_executor to enforce a 30s timeout. However, when the timeout fires, only the Future is cancelled — the underlying ThreadPoolExecutor thread running IPython.run_cell() continues indefinitely since Python cannot kill threads.

We believe this causes progressive sample hangs during AIME 2025 evaluation: 12/30 problems consistently trigger code that doesn't terminate, and over time the process stalls completely. With --concurrency 128, ~725/960 samples complete before stalling; with --concurrency 64, only ~407/960. AIME 2024 is unaffected (960/960 completes).

We haven't directly instrumented thread pool saturation, but the progressive nature of the hangs and the code path are consistent with zombie threads accumulating in the shared default ThreadPoolExecutor.

Possible fixes:

• Use ProcessPoolExecutor instead of the default ThreadPoolExecutor — processes can be killed on timeout
• Use subprocess.run with a timeout instead of IPython.run_cell in a thread
• The e2b path doesn't have this issue since sandboxes are externally killable

(Claude assisted in making this issue)