Boilerplate refactor#1

Merged

vineethcv merged 28 commits into

mainfrom

boilerplate-refactor

Apr 4, 2026

Commits on Apr 4, 2026

refactor: rename llm client concept to system under test abstraction
vineethcv
committed
feat: introduce task config as first-class evaluation input
vineethcv
committed
feat: split config into task, system, and judge configuration layers
vineethcv
committed
refactor: move wine-specific assumptions out of runner into task config
vineethcv
committed
refactor: move rubric loading and thresholds under task-driven config
vineethcv
committed
feat: add generic system client interface for pluggable app evaluation
vineethcv
committed
refactor: adapt OpenAI generation client to implement system client interface
vineethcv
committed
feat: introduce judge role abstraction for configurable evaluator perspectives
vineethcv
committed
refactor: extract judge base prompt and role overlays from judge client code
vineethcv
committed
feat: support config-driven judge prompt assembly with rubric anchors
vineethcv
committed
refactor: make judge ensemble definitions configurable by task
vineethcv
committed
feat: standardize evaluation result schema for generator, heuristics, and judges
vineethcv
committed
refactor: tag dataset cases with generic scenario metadata
vineethcv
committed
refactor: generalize dataset terminology from wine cases to evaluation cases
vineethcv
committed
feat: add boilerplate-friendly CLI options for task, system, and judge configs
vineethcv
committed
refactor: keep mock mode compatible with new boilerplate interfaces
vineethcv
committed
refactor: preserve regression compare compatibility with standardized result schema
vineethcv
committed
docs: reorganize README around boilerplate usage and architecture
vineethcv
committed
docs: add wine recommendation as reference example task
vineethcv
committed
docs: add boilerplate extension guide for new domains and judge roles
vineethcv
committed
chore: add starter config directories and example prompt templates
vineethcv
committed
test: update unit and regression tests for config-driven boilerplate flow
vineethcv
committed
chore: finalize v1 boilerplate cleanup and backward compatibility pass
vineethcv
committed
clean up
vineethcv
committed
ci: move workflows to github-hosted runners for public repo readiness
vineethcv
committed
perf: build system client once per run instead of per case
vineethcv
committed
fix: add PyYAML dependency for config loading
vineethcv
committed
ci: make baseline comparison optional when baseline file is missing
vineethcv
committed