Add CyberGym-backed cyber RL evaluation

Adds CyberBench seed generation, Z.ai smoke testing, CyberGym artifact-backed patch evaluation, Daytona SDK compatibility, and a README training workflow.

Tested: python3 -m pytest -q
Tested: python3 -m pytest -q tests/test_cybergym_artifacts.py tests/test_cyberbench_seed_pipeline.py tests/test_integrations_zai.py

Author: abhishekgahlot2

deepgym/.env.example

@@ -10,3 +10,11 @@
 
 # Anthropic (for LLM adversarial testing)
 # ANTHROPIC_API_KEY=your_key_here
+
+# Z.ai / GLM baseline model smoke tests (optional)
+# ZAI_API_KEY=your_rotated_zai_key_here
+# ZAI_API_BASE=https://api.z.ai/api/paas/v4
+# ZAI_MODEL=glm-5.1
+
+# CyberGym source dataset for seed generation (optional)
+# HF_CYBERGYM_REPO=sunblaze-ucb/cybergym

deepgym/README.md

@@ -50,6 +50,7 @@ and per-test-case breakdown showing exactly which tests passed and which didn't.
 - 2,350+ importable benchmarks (HumanEval, MBPP, BigCodeBench, EvalPlus)
 - SWE-bench Pro support for repo-level patch RL tasks
 - Terminal-Bench 2.0 support for shell/terminal RL tasks
+- CyberBench/CyberGym support for local, artifact-backed cyber patch tasks
 - MixedEnvironment routing for multi-benchmark training in one reward function
 - Per-test-case reward traces (not just pass/fail -- you see which tests broke)
 - Deterministic seeding (same input, same score, every time)
@@ -159,6 +160,74 @@ result = dg.run(
 print(result.score)
 ```
 
+### Train a cyber model with CyberBench/CyberGym
+
+DeepGym can turn CyberGym Hugging Face metadata and artifacts into
+CyberBench-style RL tasks for defensive cyber training. Use it for local
+patch-repair, log-triage, forensics, and synthetic CTF-style tasks where the
+reward comes from a deterministic verifier, not from live targets.
+
+Install the optional Hugging Face and Daytona dependencies:
+
+```bash
+pip install "deepgym[hf,daytona]"
+```
+
+Keep provider keys local and untracked:
+
+```bash
+cp .env.example .env
+# edit .env with DAYTONA_API_KEY, ZAI_API_KEY, ZAI_API_BASE, ZAI_MODEL
+```
+
+Generate safe CyberBench seed specs from CyberGym metadata. Add `--use-zai` if
+you want GLM/Z.ai to enrich the seed plans while preserving local-only safety
+constraints:
+
+```bash
+python scripts/inspect_cybergym_hf.py --repo-id sunblaze-ucb/cybergym --limit 100
+python scripts/generate_cyberbench_seeds.py --count 100 --use-zai
+```
+
+For RL reward diagnostics, prefer artifact-backed patch tasks over
+metadata-only prompts. The runner downloads the vulnerable repository archive
+and reference patch, applies a model-produced unified diff in a sandbox, then
+scores application, touched-file overlap, changed-line similarity, minimality,
+and safety scope.
+
+```bash
+# Local verifier smoke/diagnostics
+python scripts/run_cybergym_artifact_eval.py --count 100 --mode local --max-parallel 12
+
+# Daytona-isolated execution for untrusted model outputs
+python scripts/run_cybergym_artifact_eval.py --count 20 --mode daytona --max-parallel 4
+
+# Ask GLM for candidate patches when rate limits allow
+python scripts/run_cybergym_artifact_eval.py --count 20 --answer-source glm --fallback-to-reference
+```
+
+Inside a trainer, load a CyberGym row, build a `CyberGymPatchEnvironment`, and
+score model completions exactly like any other DeepGym environment:
+
+```python
+from deepgym import DeepGym
+from deepgym.cybergym_artifacts import CyberGymPatchEnvironment, load_cybergym_rows
+
+dg = DeepGym(mode='auto')  # Daytona when configured, local fallback otherwise
+row = load_cybergym_rows(count=1)[0]
+env = CyberGymPatchEnvironment.from_row(row)
+
+candidate_patch = model.generate(env.task)  # return a unified diff
+result = dg.run(env, model_output=candidate_patch)
+reward = result.score
+```
+
+Use the generated `data/cyberbench/*.jsonl` files as curriculum inputs for
+TRL/verl/OpenRLHF. Keep training and evaluation splits separate, use Daytona for
+untrusted completions, and do not train on metadata-only scores; materialize
+seed specs into artifact-backed environments with deterministic verifiers first.
+See `docs/cyberbench.md` for the full workflow and current diagnostic numbers.
+
 ### Mix multiple benchmarks behind one reward function
 
 ```python

deepgym/data/cyberbench/README.md

@@ -0,0 +1,32 @@
+# CyberBench seed data
+
+`seed_specs.jsonl` contains safe DeepGym CyberBench-RL task plans generated from
+CyberGym-style vulnerability metadata. These are curriculum seeds, not final
+weaponized exploit tasks.
+
+Regenerate deterministic seeds from Hugging Face metadata:
+
+```bash
+python scripts/generate_cyberbench_seeds.py --count 100
+```
+
+Use Z.ai/GLM enrichment after setting a rotated local key in `.env`:
+
+```bash
+python scripts/generate_cyberbench_seeds.py --count 100 --use-zai
+```
+
+Inspect CyberGym source distribution:
+
+```bash
+python scripts/inspect_cybergym_hf.py --repo-id sunblaze-ucb/cybergym --limit 200
+```
+
+## Artifact-backed evaluation outputs
+
+- `top100_artifact_eval_local.jsonl` — first 100 CyberGym rows evaluated locally
+  with reference patches through the artifact-backed verifier.
+- `top100_artifact_eval_local.summary.json` — summary of local artifact-backed
+  verifier coverage.
+- `artifact_eval_daytona_5.jsonl` — Daytona smoke run for artifact-backed patch
+  tasks.

deepgym/data/cyberbench/artifact_eval_daytona_5.jsonl

@@ -0,0 +1,5 @@
+{"index": 0, "task_id": "arvo:1065", "project": "file", "language": "c++", "answer_source": "reference", "answer_error": "", "score": 1.0, "passed": true, "details": "apply=True; file_overlap=1.00; line_similarity=1.00; safety=True", "reward_components": {"apply": 0.25, "file_overlap": 0.25, "line_similarity": 0.3, "minimality": 0.1, "safety": 0.1}, "metrics": {"candidate_files": ["src/funcs.c"], "reference_files": ["src/funcs.c"], "root": "src-vul/file"}, "cases": [{"id": "apply", "passed": true, "score": 1.0, "input_summary": "", "expected_summary": "", "actual_summary": "", "error": null, "execution_time_ms": 0.0}, {"id": "file_overlap", "passed": true, "score": 1.0, "input_summary": "", "expected_summary": "", "actual_summary": "", "error": null, "execution_time_ms": 0.0}, {"id": "line_similarity", "passed": true, "score": 1.0, "input_summary": "", "expected_summary": "", "actual_summary": "", "error": null, "execution_time_ms": 0.0}, {"id": "safety_scope", "passed": true, "score": 1.0, "input_summary": "", "expected_summary": "", "actual_summary": "", "error": null, "execution_time_ms": 0.0}], "sandbox_id": "4b4a3fcc-da69-4095-b144-1d828e63da80", "elapsed_ms": 29461.98}
+{"index": 1, "task_id": "arvo:1461", "project": "libxml2", "language": "c++", "answer_source": "reference", "answer_error": "", "score": 1.0, "passed": true, "details": "apply=True; file_overlap=1.00; line_similarity=1.00; safety=True", "reward_components": {"apply": 0.25, "file_overlap": 0.25, "line_similarity": 0.3, "minimality": 0.1, "safety": 0.1}, "metrics": {"candidate_files": ["parser.c"], "reference_files": ["parser.c"], "root": "src-vul/libxml2"}, "cases": [{"id": "apply", "passed": true, "score": 1.0, "input_summary": "", "expected_summary": "", "actual_summary": "", "error": null, "execution_time_ms": 0.0}, {"id": "file_overlap", "passed": true, "score": 1.0, "input_summary": "", "expected_summary": "", "actual_summary": "", "error": null, "execution_time_ms": 0.0}, {"id": "line_similarity", "passed": true, "score": 1.0, "input_summary": "", "expected_summary": "", "actual_summary": "", "error": null, "execution_time_ms": 0.0}, {"id": "safety_scope", "passed": true, "score": 1.0, "input_summary": "", "expected_summary": "", "actual_summary": "", "error": null, "execution_time_ms": 0.0}], "sandbox_id": "001319bd-f295-43cb-b204-1ff39bbeef96", "elapsed_ms": 87623.05}
+{"index": 2, "task_id": "arvo:65212", "project": "libssh2", "language": "c++", "answer_source": "reference", "answer_error": "", "score": 1.0, "passed": true, "details": "apply=True; file_overlap=1.00; line_similarity=1.00; safety=True", "reward_components": {"apply": 0.25, "file_overlap": 0.25, "line_similarity": 0.3, "minimality": 0.1, "safety": 0.1}, "metrics": {"candidate_files": ["src/kex.c"], "reference_files": ["src/kex.c"], "root": "src-vul/libssh2"}, "cases": [{"id": "apply", "passed": true, "score": 1.0, "input_summary": "", "expected_summary": "", "actual_summary": "", "error": null, "execution_time_ms": 0.0}, {"id": "file_overlap", "passed": true, "score": 1.0, "input_summary": "", "expected_summary": "", "actual_summary": "", "error": null, "execution_time_ms": 0.0}, {"id": "line_similarity", "passed": true, "score": 1.0, "input_summary": "", "expected_summary": "", "actual_summary": "", "error": null, "execution_time_ms": 0.0}, {"id": "safety_scope", "passed": true, "score": 1.0, "input_summary": "", "expected_summary": "", "actual_summary": "", "error": null, "execution_time_ms": 0.0}], "sandbox_id": "7446ed61-246d-4dcd-a969-9e8bce25241a", "elapsed_ms": 34241.75}
+{"index": 3, "task_id": "arvo:781", "project": "pcre2", "language": "c++", "answer_source": "reference", "answer_error": "", "score": 1.0, "passed": true, "details": "apply=True; file_overlap=1.00; line_similarity=1.00; safety=True", "reward_components": {"apply": 0.25, "file_overlap": 0.25, "line_similarity": 0.3, "minimality": 0.1, "safety": 0.1}, "metrics": {"candidate_files": ["ChangeLog", "src/pcre2_match.c"], "reference_files": ["ChangeLog", "src/pcre2_match.c"], "root": "src-vul/pcre2"}, "cases": [{"id": "apply", "passed": true, "score": 1.0, "input_summary": "", "expected_summary": "", "actual_summary": "", "error": null, "execution_time_ms": 0.0}, {"id": "file_overlap", "passed": true, "score": 1.0, "input_summary": "", "expected_summary": "", "actual_summary": "", "error": null, "execution_time_ms": 0.0}, {"id": "line_similarity", "passed": true, "score": 1.0, "input_summary": "", "expected_summary": "", "actual_summary": "", "error": null, "execution_time_ms": 0.0}, {"id": "safety_scope", "passed": true, "score": 1.0, "input_summary": "", "expected_summary": "", "actual_summary": "", "error": null, "execution_time_ms": 0.0}], "sandbox_id": "2ec44166-2916-4711-b52c-e57b7ce37ce1", "elapsed_ms": 77675.09}
+{"index": 4, "task_id": "arvo:1976", "project": "libxml2", "language": "c++", "answer_source": "reference", "answer_error": "", "score": 0.0, "passed": false, "details": "patch does not apply: valid.c: context mismatch near line 4534", "reward_components": {"apply": 0.0, "file_overlap": 0.0, "line_similarity": 0.0, "safety": 0.1}, "metrics": null, "cases": [{"id": "apply", "passed": false, "score": 0.0, "input_summary": "", "expected_summary": "", "actual_summary": "", "error": "valid.c: context mismatch near line 4534", "execution_time_ms": 0.0}], "sandbox_id": "678ace5a-4fe4-415c-abd2-87193f4d093e", "elapsed_ms": 81398.0}

deepgym/data/cyberbench/artifact_eval_daytona_5.summary.json

@@ -0,0 +1,8 @@
+{
+  "count": 5,
+  "mode": "daytona",
+  "answer_source": "reference",
+  "passed": 4,
+  "avg_score": 0.8,
+  "output": "/Users/abhishek/Desktop/New Folder With Items 2/rlgym/deepgym/data/cyberbench/artifact_eval_daytona_5.jsonl"
+}

deepgym/data/cyberbench/artifact_eval_local_3.jsonl

@@ -0,0 +1,3 @@
+{"index": 0, "task_id": "arvo:1065", "project": "file", "language": "c++", "answer_source": "reference", "answer_error": "", "score": 1.0, "passed": true, "details": "apply=True; file_overlap=1.00; line_similarity=1.00; safety=True", "reward_components": {"apply": 0.25, "file_overlap": 0.25, "line_similarity": 0.3, "minimality": 0.1, "safety": 0.1}, "metrics": {"candidate_files": ["src/funcs.c"], "reference_files": ["src/funcs.c"], "root": "src-vul/file"}, "cases": [{"id": "apply", "passed": true, "score": 1.0, "input_summary": "", "expected_summary": "", "actual_summary": "", "error": null, "execution_time_ms": 0.0}, {"id": "file_overlap", "passed": true, "score": 1.0, "input_summary": "", "expected_summary": "", "actual_summary": "", "error": null, "execution_time_ms": 0.0}, {"id": "line_similarity", "passed": true, "score": 1.0, "input_summary": "", "expected_summary": "", "actual_summary": "", "error": null, "execution_time_ms": 0.0}, {"id": "safety_scope", "passed": true, "score": 1.0, "input_summary": "", "expected_summary": "", "actual_summary": "", "error": null, "execution_time_ms": 0.0}], "sandbox_id": "local", "elapsed_ms": 1517.07}
+{"index": 1, "task_id": "arvo:1461", "project": "libxml2", "language": "c++", "answer_source": "reference", "answer_error": "", "score": 1.0, "passed": true, "details": "apply=True; file_overlap=1.00; line_similarity=1.00; safety=True", "reward_components": {"apply": 0.25, "file_overlap": 0.25, "line_similarity": 0.3, "minimality": 0.1, "safety": 0.1}, "metrics": {"candidate_files": ["parser.c"], "reference_files": ["parser.c"], "root": "src-vul/libxml2"}, "cases": [{"id": "apply", "passed": true, "score": 1.0, "input_summary": "", "expected_summary": "", "actual_summary": "", "error": null, "execution_time_ms": 0.0}, {"id": "file_overlap", "passed": true, "score": 1.0, "input_summary": "", "expected_summary": "", "actual_summary": "", "error": null, "execution_time_ms": 0.0}, {"id": "line_similarity", "passed": true, "score": 1.0, "input_summary": "", "expected_summary": "", "actual_summary": "", "error": null, "execution_time_ms": 0.0}, {"id": "safety_scope", "passed": true, "score": 1.0, "input_summary": "", "expected_summary": "", "actual_summary": "", "error": null, "execution_time_ms": 0.0}], "sandbox_id": "local", "elapsed_ms": 2764.55}
+{"index": 2, "task_id": "arvo:65212", "project": "libssh2", "language": "c++", "answer_source": "reference", "answer_error": "", "score": 1.0, "passed": true, "details": "apply=True; file_overlap=1.00; line_similarity=1.00; safety=True", "reward_components": {"apply": 0.25, "file_overlap": 0.25, "line_similarity": 0.3, "minimality": 0.1, "safety": 0.1}, "metrics": {"candidate_files": ["src/kex.c"], "reference_files": ["src/kex.c"], "root": "src-vul/libssh2"}, "cases": [{"id": "apply", "passed": true, "score": 1.0, "input_summary": "", "expected_summary": "", "actual_summary": "", "error": null, "execution_time_ms": 0.0}, {"id": "file_overlap", "passed": true, "score": 1.0, "input_summary": "", "expected_summary": "", "actual_summary": "", "error": null, "execution_time_ms": 0.0}, {"id": "line_similarity", "passed": true, "score": 1.0, "input_summary": "", "expected_summary": "", "actual_summary": "", "error": null, "execution_time_ms": 0.0}, {"id": "safety_scope", "passed": true, "score": 1.0, "input_summary": "", "expected_summary": "", "actual_summary": "", "error": null, "execution_time_ms": 0.0}], "sandbox_id": "local", "elapsed_ms": 2460.0}

deepgym/data/cyberbench/artifact_eval_local_3.summary.json

@@ -0,0 +1,8 @@
+{
+  "count": 3,
+  "mode": "local",
+  "answer_source": "reference",
+  "passed": 3,
+  "avg_score": 1.0,
+  "output": "/Users/abhishek/Desktop/New Folder With Items 2/rlgym/deepgym/data/cyberbench/artifact_eval_local_3.jsonl"
+}