You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Extract shared scorer utilities, move AgentDojo tools, refresh Chinese README
- Extract _run_coro_sync() to snowl/scorer/_sync_bridge.py (dedup model_judge + grade_judge)
- Extract render_judge_prompt() to snowl/scorer/_prompt.py (dedup template rendering)
- Move AgentDojo banking/travel tools to snowl/benchmarks/agentdojo/tools.py (P1.5 fix)
- Update fortress and toolemu scorers to use shared _sync_bridge
- Full Chinese README refresh aligned with English version
- Update governance.md: mark P1.3, P1.5, P2.1, P2.2 as RESOLVED
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|**MEDIUM**|`snowl/runtime/container_providers.py:25`| Runtime imports `OSWorldContainerLauncher` from a specific benchmark |
108
-
|**MEDIUM**|`snowl/runtime/engine.py:961`| Engine checks `output.get("osworld_score")` — benchmark-specific logic in generic engine|
109
-
|**LOW**|`snowl/tools/stateful_executor.py:53-408`| AgentDojo-specific tool implementations in shared `snowl/tools/` package |
108
+
|**MEDIUM**|`snowl/runtime/engine.py:961`|~~Engine checks `output.get("osworld_score")`~~**RESOLVED**: replaced with generic `_get_extra_payload_keys()` that reads from benchmark registry|
109
+
|**LOW**|`snowl/tools/stateful_executor.py:53-408`|~~AgentDojo-specific tool implementations in shared `snowl/tools/` package~~**RESOLVED**: moved to `snowl/benchmarks/agentdojo/tools.py`|
110
110
|**LOW**|`snowl/benchmarks/base_adapter.py:46-48`|`benchmark_info()` does deferred import of registry (semantic coupling) |
111
111
112
112
### 1.5 Duplicate Abstractions
113
113
114
114
| Duplication | Locations | Fix |
115
115
|-------------|-----------|-----|
116
-
|`_run_coro_sync()`|`scorer/model_judge.py:30-52`, `scorer/grade_judge.py:20-40`| Extract to `scorer/_sync_bridge.py`|
117
-
| Template rendering |`scorer/model_judge.py` (`_render_template`), `scorer/grade_judge.py` (`render_prompt_template`)| Unify to single implementation in `scorer/grade_judge.py`|
116
+
|`_run_coro_sync()`|~~`scorer/model_judge.py:30-52`, `scorer/grade_judge.py:20-40`~~**RESOLVED**: extracted to `scorer/_sync_bridge.py`|
117
+
| Template rendering |~~`scorer/model_judge.py` (`_render_template`), `scorer/grade_judge.py` (`render_prompt_template`)~~**RESOLVED**: unified to `scorer/_prompt.py:render_judge_prompt()`|
118
118
|`_tool_schemas()`|`benchmarks/bfcl/adapter.py`, `benchmarks/agentdojo/adapter.py`| Extract to `benchmarks/utils.py`|
#### P1.3: Remove benchmark-specific logic from runtime engine
345
+
#### P1.3: ~~Remove benchmark-specific logic from runtime engine~~ RESOLVED
346
346
347
-
-**Problem**: `engine.py:961` checks `output.get("osworld_score")` — benchmark-specific in generic engine
347
+
-**Problem**: ~~`engine.py:961` checks `output.get("osworld_score")` — benchmark-specific in generic engine~~
348
+
-**Resolution**: Replaced with generic `_get_extra_payload_keys()` that reads key names from benchmark registry at runtime. No benchmark-specific key names hardcoded in engine.
348
349
-**Files**: `snowl/runtime/engine.py`
349
-
-**Why it matters**: Every benchmark shouldn't need engine modifications
350
-
-**Violates core/adapters?**: Yes — runtime depends on benchmark details
351
-
-**Risk level**: Medium
352
-
-**Strategy**: Move OSWorld score extraction to OSWorld's scorer; use generic payload passthrough in engine
0 commit comments