Skip to content

Commit 2112ab4

Browse files
committed
Add pre-HN readiness toolkit
1 parent e70c78e commit 2112ab4

16 files changed

Lines changed: 438 additions & 3 deletions

File tree

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
# Evidence - pre-hn-readiness
2+
3+
## Build Summary
4+
5+
Implemented the five pre-HN readiness items:
6+
7+
- added `Makefile` with `make test`
8+
- added stdlib unittest coverage under `tests/`
9+
- added executable wrappers `bin/proof-loop-init` and `bin/proof-loop-check`
10+
- added realistic demo fixture under `examples/demo-repo/`
11+
- added GitHub Actions workflow `.github/workflows/test.yml`
12+
- updated README with test badge, `20-second demo`, and `What This Is Not`
13+
14+
## Checks Run
15+
16+
```bash
17+
make test
18+
```
19+
20+
Result:
21+
22+
```text
23+
Ran 7 tests in 0.398s
24+
OK
25+
NAV_LABEL_CHECK_PASS
26+
PROOF_LOOP_PASS examples/example-task/.agent/tasks/ui-language-fix
27+
PROOF_LOOP_PASS examples/demo-repo/.agent/tasks/nav-labels-proof
28+
```
29+
30+
```bash
31+
cd /tmp/proof-loop-wrapper-smoke/other-repo
32+
/home/less/repo-garden-audit/repos/proof-loop/bin/proof-loop-init external-hn --title "External HN smoke" --root .
33+
/home/less/repo-garden-audit/repos/proof-loop/bin/proof-loop-check .agent/tasks/external-hn
34+
```
35+
36+
Result: wrapper init created `.agent/tasks/external-hn/`; wrapper check failed as expected because the task was unverified.
37+
38+
```bash
39+
python3 examples/demo-repo/check_nav_labels.py examples/demo-repo/nav_labels.json
40+
python3 scripts/check_task.py examples/demo-repo/.agent/tasks/nav-labels-proof
41+
```
42+
43+
Result: demo check passed and the demo proof task passed.
44+
45+
## AC1 - PASS
46+
47+
`make test` passes and covers initialization, invalid IDs, unverified failure, completed example pass, invalid JSON, non-empty `problems.md`, and wrappers from another repo.
48+
49+
## AC2 - PASS
50+
51+
`bin/proof-loop-init` and `bin/proof-loop-check` work from a temporary directory outside this repository.
52+
53+
## AC3 - PASS
54+
55+
`examples/demo-repo/` contains a real JSON fixture, a real checker script, and matching proof artifacts. The checker and task gate both pass.
56+
57+
## AC4 - PASS
58+
59+
`.github/workflows/test.yml` runs `make test`; README includes the workflow badge.
60+
61+
## AC5 - PASS
62+
63+
README includes `20-second demo` and `What This Is Not`; commands shown are backed by committed wrappers, Makefile, and examples.

.agent/tasks/pre-hn-readiness/problems.md

Whitespace-only changes.
Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# Task: pre-hn-readiness
2+
3+
## Task Statement
4+
5+
Make Proof Loop ready for a Show HN audience by turning it from a useful protocol repo into a small, tryable, verified tool with tests, CI, executable wrappers, a realistic demo fixture, and a sharper README entry path.
6+
7+
## Acceptance Criteria
8+
9+
**AC1:** The repository has a real test suite and a single `make test` command that exercises the helper scripts.
10+
- Verify: run `make test`; it must pass and cover task initialization, invalid task IDs, unverified FAIL, completed PASS, invalid JSON, and non-empty `problems.md`.
11+
12+
**AC2:** Proof Loop can be invoked from another repository through executable wrapper commands, not only by running `python3 scripts/...` inside the cloned repo.
13+
- Verify: from a temporary directory outside this repo, run `bin/proof-loop-init` and `bin/proof-loop-check` using absolute paths; initialization must create `.agent/tasks/<TASK_ID>/` in that external directory and the check must fail until the task is verified.
14+
15+
**AC3:** The repo includes a realistic demo fixture whose proof artifacts correspond to an actual file and check command.
16+
- Verify: run the demo check command documented in the demo evidence; it must pass against files in `examples/demo-repo/`, and the demo task must pass `scripts/check_task.py`.
17+
18+
**AC4:** GitHub Actions CI is present and the README exposes the test badge.
19+
- Verify: `.github/workflows/test.yml` exists, runs `make test`, and README contains the workflow badge.
20+
21+
**AC5:** The README has an HN-ready top section: a 20-second demo and a clear `What This Is Not` section.
22+
- Verify: README contains `20-second demo` and `What This Is Not`, and the commands shown in the demo are backed by actual files/scripts.
23+
24+
## Constraints
25+
26+
- Keep runtime dependencies to Python standard library.
27+
- Do not introduce packaging complexity unless needed.
28+
- Do not remove existing examples, role briefs, or references.
29+
- Keep claims harness-agnostic and honest; do not imply native integrations.
30+
31+
## Non-Goals
32+
33+
- Publishing to PyPI.
34+
- Building a full web demo.
35+
- Creating an umbrella repo or platform brand.
36+
- Adding Loopsmith eval packs in this task.
37+
38+
## Verification Approach
39+
40+
Run `make test`, run the wrappers from outside the repo, run the realistic demo check, run `scripts/check_task.py` against the demo proof task, and finally update `verdict.json` so every AC is PASS only after those checks succeed.
Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
{
2+
"task_id": "pre-hn-readiness",
3+
"phase": "verify",
4+
"agent": "francis-fresh-verifier",
5+
"timestamp": "2026-05-20T15:55:00Z",
6+
"overall": "PASS",
7+
"criteria": [
8+
{
9+
"id": "AC1",
10+
"status": "PASS",
11+
"note": "make test passes and covers init, invalid IDs, unverified FAIL, completed PASS, invalid JSON, non-empty problems.md, and wrapper usage."
12+
},
13+
{
14+
"id": "AC2",
15+
"status": "PASS",
16+
"note": "Absolute bin/proof-loop-init and bin/proof-loop-check were run from /tmp/proof-loop-wrapper-smoke/other-repo; init created artifacts and check failed until verification as expected."
17+
},
18+
{
19+
"id": "AC3",
20+
"status": "PASS",
21+
"note": "examples/demo-repo has a real JSON fixture and checker; checker passes and the demo proof task passes check_task.py."
22+
},
23+
{
24+
"id": "AC4",
25+
"status": "PASS",
26+
"note": ".github/workflows/test.yml runs make test and README contains the test badge."
27+
},
28+
{
29+
"id": "AC5",
30+
"status": "PASS",
31+
"note": "README contains 20-second demo and What This Is Not; demo commands reference committed wrappers, Makefile, and examples."
32+
}
33+
]
34+
}

.github/workflows/test.yml

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
name: test
2+
3+
on:
4+
push:
5+
pull_request:
6+
7+
jobs:
8+
test:
9+
runs-on: ubuntu-latest
10+
steps:
11+
- uses: actions/checkout@v4
12+
- uses: actions/setup-python@v5
13+
with:
14+
python-version: "3.x"
15+
- run: make test

Makefile

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
.PHONY: test
2+
3+
test:
4+
python3 -m py_compile scripts/init_task.py scripts/check_task.py examples/demo-repo/check_nav_labels.py
5+
python3 -m unittest discover -s tests
6+
python3 examples/demo-repo/check_nav_labels.py examples/demo-repo/nav_labels.json
7+
python3 scripts/check_task.py examples/example-task/.agent/tasks/ui-language-fix
8+
python3 scripts/check_task.py examples/demo-repo/.agent/tasks/nav-labels-proof

README.md

Lines changed: 41 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,34 @@
11
# Proof Loop
22

3+
![Tests](https://github.com/LeoStehlik/proof-loop/actions/workflows/test.yml/badge.svg)
4+
35
**Finish AI coding work with evidence, not vibes.**
46

57
Proof Loop is a lightweight protocol and toolkit for non-trivial coding tasks handled by AI agents. It freezes acceptance criteria before the build, separates builder and verifier roles, records durable proof artifacts in the repo, and refuses to call work done until every acceptance criterion has a fresh PASS verdict.
68

79
Use it when an agent, team, or multi-agent sprint needs a hard boundary against false completion claims. Because the protocol is just files plus role discipline, it works with OpenClaw, Hermes, Codex, OpenCode, Claude Code, or any other harness that can read and write a repository.
810

11+
12+
## 20-second demo
13+
14+
```bash
15+
git clone https://github.com/LeoStehlik/proof-loop.git
16+
cd proof-loop
17+
make test
18+
19+
tmp=$(mktemp -d)
20+
bin/proof-loop-init hn-demo --title "Prove this task before done" --root "$tmp"
21+
bin/proof-loop-check "$tmp/.agent/tasks/hn-demo"
22+
```
23+
24+
The last command fails on purpose because the generated task has not been verified yet. Proof Loop only returns success after a fresh verifier records `PASS` for every acceptance criterion and `problems.md` is empty.
25+
26+
A completed passing example is included:
27+
28+
```bash
29+
bin/proof-loop-check examples/example-task/.agent/tasks/ui-language-fix
30+
```
31+
932
## Why It Exists
1033

1134
AI coding agents often fail in predictable ways:
@@ -31,10 +54,10 @@ Proof Loop makes completion auditable. A task is done only when a fresh verifier
3154

3255
Clone the repo or copy it into the project where you want to run the protocol.
3356

34-
Create a task proof folder:
57+
Create a task proof folder from this repo or from another repository:
3558

3659
```bash
37-
python3 scripts/init_task.py ui-language-fix --title "Fix German navigation labels"
60+
bin/proof-loop-init ui-language-fix --title "Fix German navigation labels" --root .
3861
```
3962

4063
This creates:
@@ -52,7 +75,7 @@ Fill `spec.md` with explicit acceptance criteria before implementation starts.
5275
After the build and verifier pass, check whether the task is allowed to be called done:
5376

5477
```bash
55-
python3 scripts/check_task.py .agent/tasks/ui-language-fix
78+
bin/proof-loop-check .agent/tasks/ui-language-fix
5679
```
5780

5881
The check exits non-zero unless:
@@ -61,6 +84,16 @@ The check exits non-zero unless:
6184
- every AC has `status: PASS`
6285
- `problems.md` is empty or absent
6386

87+
88+
## What This Is Not
89+
90+
- not an agent framework
91+
- not a benchmark suite
92+
- not a replacement for tests
93+
- not tied to one model, vendor, or harness
94+
95+
Proof Loop is deliberately small: a protocol, a few files, and a mechanical done gate.
96+
6497
## The Protocol
6598

6699
```text
@@ -186,9 +219,14 @@ Copy the `proof-loop` folder into your agent skills directory, or reference `SKI
186219
```text
187220
proof-loop/
188221
SKILL.md skill trigger and core operating rules
222+
bin/
223+
proof-loop-init wrapper usable from another repository
224+
proof-loop-check wrapper usable from another repository
189225
scripts/
190226
init_task.py create .agent/tasks/<TASK_ID>/ skeletons
191227
check_task.py mechanical done gate
228+
tests/ stdlib unittest coverage for CLI behavior
229+
.github/workflows/test.yml CI running make test
192230
references/
193231
workflow.md full phase-by-phase protocol
194232
brief-template.md reusable sprint and role prompts

bin/proof-loop-check

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
#!/usr/bin/env sh
2+
set -eu
3+
SCRIPT_DIR=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
4+
exec python3 "$SCRIPT_DIR/../scripts/check_task.py" "$@"

bin/proof-loop-init

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
#!/usr/bin/env sh
2+
set -eu
3+
SCRIPT_DIR=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
4+
exec python3 "$SCRIPT_DIR/../scripts/init_task.py" "$@"
Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
# Evidence - nav-labels-proof
2+
3+
## Build Summary
4+
5+
The demo fixture contains `examples/demo-repo/nav_labels.json` and a real check command at `examples/demo-repo/check_nav_labels.py`.
6+
7+
## Checks Run
8+
9+
```bash
10+
python3 examples/demo-repo/check_nav_labels.py examples/demo-repo/nav_labels.json
11+
```
12+
13+
Result:
14+
15+
```text
16+
NAV_LABEL_CHECK_PASS
17+
```
18+
19+
## AC1 - PASS
20+
21+
The JSON file contains the expected English labels: Home, Settings, Billing.
22+
23+
## AC2 - PASS
24+
25+
The JSON file contains the expected German labels: Startseite, Einstellungen, Abrechnung.
26+
27+
## AC3 - PASS
28+
29+
The check script compares the file against an expected dictionary and exits with status `1` when values differ.

0 commit comments

Comments
 (0)