Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
996523f
auth: add User-Agent header + PlatformUnavailableError on non-JSON re…
cleverhoods May 12, 2026
e103f62
Add per-capability targeting + focus/listing/top-rules to ails check
cleverhoods May 17, 2026
481cf04
Add configurable rule thresholds + generic file class via Markdown li…
cleverhoods May 17, 2026
005fd7e
Fix non-deterministic message text in `client_checks._check_broad_scope`
cleverhoods May 17, 2026
a1e7a7a
Fix `detect_features_filesystem` counting user-scope CLAUDE.md as pro…
cleverhoods May 17, 2026
891d7ef
Fix memory/subagent_memory discovery enumerating *.md inside director…
cleverhoods May 18, 2026
437b959
Add CORE:S:0056 broken-markdown-link rule + thread CheckResult.annota…
cleverhoods May 18, 2026
4336d54
Add memory_locator.py — per-agent memory entry locator used by L3 mem…
cleverhoods May 18, 2026
a45f62b
Fix mapper daemon attach: socket-existence gate, FIFO-only FD close, …
cleverhoods May 18, 2026
21d2b8f
Widen CORE:S:0024 + CORE:S:0056 match scope to [freeform, frontmatter]
cleverhoods May 18, 2026
f0922c5
discovery: Sync per-agent memory file_types with upstream (gemini dir…
cleverhoods May 18, 2026
9bf1ddc
classify: attribute link-reached generic files with source verb + lin…
cleverhoods May 18, 2026
61b2dea
docs: backfill 0.5.10 surfaces — capability focus, per-rule threshold…
cleverhoods May 18, 2026
62df2d0
tests: wrap E501 docstring in test_symlink_detection TestWalkGlobFoll…
cleverhoods May 18, 2026
9e4fc4b
agents: exclude CORE:S:0024 import-targets-resolve for codex+copilot …
cleverhoods May 18, 2026
fbd674e
rules: bump severity medium→high for broken-link / import rules
cleverhoods May 18, 2026
dda9c36
classify: capability listing honors exclude_dirs, delegates memory to…
cleverhoods May 18, 2026
23ac32e
check: capability focus/listing reuse whole-repo display via filtered…
cleverhoods May 18, 2026
e30debe
check: strip focus/listing concept — one display, capability args fil…
cleverhoods May 18, 2026
9cab118
check: filter quality.compliance_band to subset + suppress duplicate …
cleverhoods May 18, 2026
7772dbf
check: per-item health bars for capability listings — name + bar per …
cleverhoods May 18, 2026
d7ef46d
check: finding-count breakdown (N: Xe/Yw/Zi) after each item health s…
cleverhoods May 18, 2026
9c2d7d1
check: item health bars one-per-line — drop 2-column layout + trailin…
cleverhoods May 19, 2026
e646f2b
check: split score bar markup — colored fill + dim gray empty, shared…
cleverhoods May 19, 2026
58aecc6
check: blank line between item-health severity bands (red/yellow/gree…
cleverhoods May 19, 2026
c2c9d25
Fix lint pipeline scope: code-span strip in extractors, exclude_dirs …
cleverhoods May 19, 2026
6a948aa
Add capability level display to scorecard — re-align engine to L0–L7 …
cleverhoods May 19, 2026
47b5475
Cover 0.5.10 lint-pipeline fixes with unit tests; extract per-item sc…
cleverhoods May 19, 2026
ed9413a
Release 0.5.10
cleverhoods May 19, 2026
b84f37a
lint: gate ~/ rendering on classifier precedence, not path-prefix
cleverhoods May 19, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions .githooks/pre-commit
Original file line number Diff line number Diff line change
@@ -1,5 +1,19 @@
#!/usr/bin/env bash
# Pre-commit hook: require UNRELEASED.md when staging src/ or tests/ changes.
# Skipped during rebase or amend operations — those rewrite existing history
# rather than introducing new user-facing changes, so a new UNRELEASED entry
# is not warranted (any user-facing entry was added by the original commit
# being rewritten).

# Skip during interactive or non-interactive rebase
if [ -d .git/rebase-merge ] || [ -d .git/rebase-apply ]; then
exit 0
fi

# Skip during amend (reflog action set by git when amending)
case "$GIT_REFLOG_ACTION" in
*amend*) exit 0 ;;
esac

staged=$(git diff --cached --name-only)

Expand Down
53 changes: 53 additions & 0 deletions CHANGELOG.md

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Reporails CLI (v0.5.9)
# Reporails CLI (v0.5.10)

> **AI Instruction Diagnostics for coding agents. Validates the entire agentic instruction system against 120+ rules across six rule packs (core + per-agent). Supports Claude, Codex, Copilot, Cursor, and Gemini.**
>
Expand Down
2 changes: 2 additions & 0 deletions UNRELEASED.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,6 @@

### Fixed

- [Lint]: Gate user-scope `~/...` rendering in mechanical-check attribution on the classifier's `precedence: user` property (read from agent config patterns) instead of path-prefix heuristics, so Windows tmp paths under the user profile no longer render with a `~/` prefix.

### Removed
6 changes: 3 additions & 3 deletions docs/agent-support.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
---
title: "Agent Support"
description: "Which agents are recognized and what's covered"
version: "0.5.6"
last_updated: 2026-05-04
version: "0.5.10"
last_updated: 2026-05-18
---

# Agent Support
Expand All @@ -17,7 +17,7 @@ Reporails recognizes the instruction-file conventions of five coding agents and
| Codex | `AGENTS.md` (+ optional `AGENTS.override.md`) | `.codex/rules/*.rules` | `.agents/skills/**/SKILL.md` | `.codex/agents/*.toml` | hooks, `.codex/config.toml`, skill metadata (`agents/openai.yaml`) |
| Copilot (VS Code) | `.github/copilot-instructions.md` or `**/AGENTS.md` | `.github/instructions/**/*.instructions.md`, `.claude/rules/**/*.md` | `.github/skills/`, `.claude/skills/`, `.agents/skills/` | `.github/agents/*.agent.md` | hooks, prompts, MCP |
| Cursor | `**/AGENTS.md` (`.cursorrules` recognized but legacy) | `.cursor/rules/**/*.mdc`, `.cursor/rules/**/*.md` | `.cursor/skills/`, `.claude/skills/`, `.codex/skills/` | `.cursor/agents/*.md`, `.claude/agents/*.md`, `.codex/agents/*.md` | hooks, MCP, managed policy, bugbot rules |
| Gemini | `GEMINI.md` or `**/AGENTS.md` | (no dedicated rules surface) | `.gemini/skills/**/SKILL.md` | `.gemini/agents/*.md` | commands, extensions, settings, hooks |
| Gemini | `GEMINI.md` or `**/AGENTS.md` | (no dedicated rules surface) | `.gemini/skills/**/SKILL.md` | `.gemini/agents/*.md` | commands, extensions, settings, hooks, memory (section inside `~/.gemini/GEMINI.md`), MCP, system_prompt, geminiignore |

Many agents intentionally read each other's directories — Cursor's skills column, for example, includes `.claude/skills/` and `.codex/skills/` because Cursor invokes skills regardless of which agent first authored them. The cells above show the most common project-level patterns; user-level and system-level patterns are also recognized — see [What gets scanned](#what-gets-scanned).

Expand Down
68 changes: 68 additions & 0 deletions docs/capability-levels.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
---
title: "Capability Levels"
description: "The ladder for where AI instructions live and how they act"
version: "0.5.10"
last_updated: 2026-05-19
---

# Capability Levels

`ails check` reports a `Level: L# <Name>` line in the scorecard between `Agent:` and `Scope:`. The level is a read-out, not a gate — every rule fires when its match conditions apply regardless of which level your project is at. Use the level to self-locate; use the symptom table at the bottom to decide when to climb.

## The Ladder

| Level | Name | What's added | Channel |
|-------|------------|-----------------------------------------------------------------|-----------------------|
| L0 | System | System prompt only | attention |
| L1 | Primer | One instruction file (`CLAUDE.md`, `AGENTS.md`, `.cursorrules`) | attention |
| L2 | Composite | Multiple files — user defaults, project overrides | attention |
| L3 | Scoped | Path-scoped rules (`.claude/rules/*.md`) | attention |
| L4 | Delegated | Skills — procedures invoked on demand | attention |
| L5 | Abstracted | Sub-agents — child contexts called by the parent | attention (interface) |
| L6 | Governed | Hooks, MCP gates, deny-permissions | enforcement |
| L7 | Adaptive | Self-improving skills written by the agent | self-writing |

The ladder sorts by the channel each rung runs on: soft attention (L0–L5), hard enforcement (L6), self-writing memory (L7). Each rung adds a new diagnostic concern — scope leakage at L3, skill-instruction coherence at L4, governance-instruction alignment at L6, drift detection at L7.

## Channels

- **Attention** — text the model reads and weights against everything else loaded. Fails probabilistically; competes for budget; decays with load. Fixes are content and ordering.
- **Enforcement** — hooks, MCP gates, deny-permissions. Acts outside the model's context. Fails deterministically when configured wrong, never silently. Fixes are scripts, schemas, permission rules.
- **Self-writing** — agent-authored instructions written between sessions. At read time these land in attention like anything else; at write time the user never saw the prompt that produced them. Fixes are review cadence and explicit auto-memory boundaries.

## Detection

The displayed level is the highest architectural capability present, cumulative — every level below must also pass.

| Detected | Level |
|----------------------------------------------------|------------------|
| Auto-memory, learned rules | L7 (Adaptive) |
| Hooks, MCP servers, managed policies | L6 (Governed) |
| Sub-agent definitions | L5 (Abstracted) |
| Skill definitions | L4 (Delegated) |
| Path-scoped rules (`.claude/rules/` with `paths:`) | L3 (Scoped) |
| Multiple main files, user defaults, overwrites | L2 (Composite) |
| Single main instruction file | L1 (Primer) |
| No instruction files | L0 (System) |

A project with skills but no path-scoped rules still displays L4 — the gate engine walks L1 → L2 → ... → L7 cumulatively. Hub-only repos that ship hooks but no instructions sit at L0 with an enforcement layer; the level reads the soft channel.

## When to climb

Each rung exists because the rung below it fails in a specific way. The trigger is the failure, not a feature wishlist.

| From | To | Symptom that triggers the climb |
|------|----|-----------------------------------------------------------------------------------------|
| L0 | L1 | Re-explaining the same project context every session |
| L1 | L2 | One file got long enough that important rules get ignored |
| L2 | L3 | Path-irrelevant rules pollute every task |
| L3 | L4 | The same procedure gets described inline across multiple rules |
| L4 | L5 | A procedure pollutes the parent's context with reasoning chains the parent doesn't need |
| L5 | L6 | A constraint must hold 100% of the time, not 95% |
| L6 | L7 | You keep correcting the same preference across sessions |

Climbing without a symptom adds structure the model has to navigate without solving a problem you had. Under-climbing is more common: *"agent didn't run tests before pushing"* reads like a prompt-engineering problem but is usually a missing L6 hook; *"agent forgot we use Cloudflare Workers"* reads like context drift but is usually a missing L7 memory entry.

---

[← Score Guide](score-guide.md) · Capability Levels · [FAQ →](faq.md)
31 changes: 29 additions & 2 deletions docs/configuration.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
---
title: "Configuration"
description: "Disabling rules, project / global config, exclude paths"
version: "0.5.7"
last_updated: 2026-05-06
version: "0.5.10"
last_updated: 2026-05-18
---

# Configuration
Expand Down Expand Up @@ -154,6 +154,33 @@ When `ails config set …` writes `.ails/config.yml`, it also writes `.ails/.git
config.local.yml
```

## Per-rule thresholds

Some rules ship with a built-in `min_lines` gate so small files do not get flagged for issues that only matter at scale. For example, `CORE:S:0013 scope-fields-in-frontmatter` ships with `min_lines: 30` — a 5-line rule file won't fail it. You can raise or lower the threshold per project under `overrides.rule_thresholds`:

```yaml
# .ails/config.yml
overrides:
rule_thresholds:
CORE:S:0013:
min_lines: 50 # require 50+ lines before this rule fires
CORE:C:0034:
min_lines: 0 # always fire, even on tiny files
```

Any deterministic check that declares a `min_lines:` entry in its `checks.yml` can be tuned this way — see `ails explain <rule_id>` for which rules expose the gate.

## Generic-class scanning (opt-in)

By default, `ails check` only validates files that match one of the agent's declared instruction-file patterns. Set `generic_scanning: true` to extend coverage to any reachable Markdown file:

```yaml
# .ails/config.yml
generic_scanning: true
```

When on, the discovery walker follows inline and reference Markdown links out of classified files (bounded depth, cycle-safe, tree-bound), and any reached `.md` file inside your repo gets a `generic` classification. Structural and formatting rules (charge ordering, direction imbalance, formatting hygiene) still fire on these files; main-shape rules (tech stack, MCP docs) do not. Default is off so anonymous tryouts against third-party repos stay quiet.

## Severity overrides

Severity is what makes a finding "critical" vs "info". Default severity comes from the rule itself; you can override it per project:
Expand Down
2 changes: 1 addition & 1 deletion docs/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,4 +78,4 @@ Open an issue at [github.com/reporails/cli/issues](https://github.com/reporails/

---

[← Score Guide](score-guide.md) · FAQ
[← Capability Levels](capability-levels.md) · FAQ
20 changes: 18 additions & 2 deletions docs/getting-started.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
---
title: "Getting Started"
description: "Install, first run, what the output means"
version: "0.5.6"
last_updated: 2026-05-04
version: "0.5.10"
last_updated: 2026-05-18
---

# Getting Started
Expand Down Expand Up @@ -105,6 +105,22 @@ ails check --agent claude # only run rules scoped to one agent

The JSON output groups findings under `files{path: {findings: [...], count: N}}` plus aggregate `stats` and (when present) `cross_file` blocks — see [Configuration → Output format](configuration.md#output-format) for the full shape, including which fields are tier-conditional.

## Focus on one file or capability

When the whole-repo view is too noisy, name the capability and (optionally) the target:

```bash
ails check skill backlog # focus on .claude/skills/backlog/SKILL.md
ails check rule git # focus on .claude/rules/git.md
ails check agent rule-writer
# subagent + any skills its frontmatter preloads
ails check skill # listing mode — table of all skills with scores
```

The full pipeline still runs (so cross-file rules see the whole project), but only the focused file or capability appears in the output, with findings grouped by rule and a `Next:` action pointer. Listing mode (`ails check <capability>` with no name) prints a per-target score table for that capability under the detected agent. Capability names come from the agent's declared `file_types:` — both singular and plural are accepted.

The whole-repo summary also shows a `Top rules (by finding count)` block — a fast triage view of which rule classes contribute the most findings across your project.

## Next steps

- [Score Guide](score-guide.md) — what the number means in practice
Expand Down
2 changes: 1 addition & 1 deletion docs/score-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,4 +76,4 @@ Score moves should be small commit-to-commit. A sudden drop usually means you re

---

[← Configuration](configuration.md) · Score Guide · [FAQ →](faq.md)
[← Configuration](configuration.md) · Score Guide · [Capability Levels →](capability-levels.md)
31 changes: 0 additions & 31 deletions framework/registry/levels.yml

This file was deleted.

4 changes: 4 additions & 0 deletions framework/rules/claude/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -271,6 +271,9 @@ file_types:
maintainer: system

memory:
# Memory directory holds MEMORY.md (index, eager) + sibling *.md topic
# files (read on-demand). Directory glob matches both with shared
# `loading: session_start`; per-entry loading split is a future move.
source: https://code.claude.com/docs/en/memory#auto-memory
format: freeform
scope: global
Expand All @@ -288,6 +291,7 @@ file_types:
# Per-subagent persistent memory. Scope chosen via `memory:` frontmatter field
# on the agent definition (user|project|local) — mutually exclusive per subagent.
# MEMORY.md (first 200 lines / 25KB) injected into subagent system prompt at startup.
# Same MEMORY.md-vs-sibling loading-granularity caveat as `memory` above.
source: https://code.claude.com/docs/en/sub-agents#enable-persistent-memory
format: freeform
scope: task_scoped
Expand Down
2 changes: 1 addition & 1 deletion framework/rules/claude/import-depth-within-limit/rule.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ slug: import-depth-within-limit
title: Import Depth Within Limit
category: structure
type: mechanical
severity: medium
severity: high
match: {type: main}
supersedes: CORE:S:0033
source: https://code.claude.com/docs/en/memory#import-additional-files
Expand Down
28 changes: 28 additions & 0 deletions framework/rules/codex/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -230,6 +230,33 @@ file_types:
vcs: external
maintainer: human

memory:
# Codex memory (`~/.codex/memories/`) is generated state, not a
# user-authored markdown surface. Per the official docs: "Codex stores
# memories under your Codex home directory" and "treats these files as
# generated state" — they're populated in the background and shaped by
# the model, not by the user.
#
# Control plane (not directly inspectable on disk):
# - `/memories` slash command in the TUI/app — per-thread enable for
# reading existing memories and feeding new ones.
# - `config.toml` keys: `memories.generate_memories`,
# `memories.use_memories`, `memories.disable_on_external_context`,
# `memories.min_rate_limit_remaining_percent`, `memories.extract_model`,
# `memories.consolidation_model`.
#
# Declared as a tombstone — no `scopes:` block, no patterns to glob.
# This keeps the surface visible in the agent registry (so audits know
# it exists) while inviting no instruction-quality rule pressure.
# `maintainer: agent` and `lifecycle: mutable` mark it as agent-owned
# cache, not human-authored content.
source: https://developers.openai.com/codex/memories
format: opaque
scope: global
cardinality: collection
lifecycle: mutable
loading: session_start

enterprise:
# Admin-enforced requirements.toml. Constrains approval policy, sandbox,
# MCP allowlists, hooks, command rules, filesystem permissions.
Expand All @@ -251,4 +278,5 @@ file_types:
excludes:
- CLAUDE:*
- COPILOT:*
- CORE:S:0024 # import-targets-resolve — Codex AGENTS.md does not honor @<path> import syntax (single-file format per developers.openai.com/codex/guides/agents-md)
- CORE:S:0033 # import-depth-within-limit — Codex AGENTS.md does not support chained @import
1 change: 1 addition & 0 deletions framework/rules/copilot/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -190,4 +190,5 @@ file_types:
excludes:
- CLAUDE:*
- CODEX:*
- CORE:S:0024 # import-targets-resolve — Copilot instruction files have no documented @<path> import syntax (per docs.github.com/en/copilot/how-tos/configure-custom-instructions/add-repository-instructions)
- CORE:S:0033 # import-depth-within-limit — Copilot instructions files do not support chained @import
14 changes: 14 additions & 0 deletions framework/rules/core/broken-markdown-link/checks.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
checks:
- id: CORE.S.0056.file-in-scope
type: mechanical
check: file_exists
args:
path: "**/*.md"
- id: CORE.S.0056.extract-links
type: mechanical
check: extract_markdown_links
args:
path: "**/*.md"
- id: CORE.S.0056.targets-exist
type: mechanical
check: check_markdown_link_targets_exist
49 changes: 49 additions & 0 deletions framework/rules/core/broken-markdown-link/rule.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
---
id: CORE:S:0056
slug: broken-markdown-link
title: Markdown Link Targets Resolve
category: structure
type: mechanical
severity: high
backed_by: []
match: {format: [freeform, frontmatter]}
---

# Markdown Link Targets Resolve

Markdown links in instruction files must resolve to existing files. Broken targets create phantom context — the agent sees the link directive but the referenced content never loads, so the file silently underdelivers compared to what its prose promises.

## Antipatterns

- **Renamed file without updating the link**: Moving `docs/setup.md` to `docs/getting-started.md` but leaving `[Setup](docs/setup.md)` in another file. The `extract_markdown_links` check finds the reference and `check_markdown_link_targets_exist` fails because the path no longer resolves.
- **Typo in relative path**: Writing `[Rules](.claude/rules/git-rules.md)` instead of `[Rules](.claude/rules/git.md)`. The link silently fails — the surrounding prose still reads as if the target loaded.
- **Link crossing repository boundary**: Referencing `[Notes](../../other-repo/CLAUDE.md)` which exists in the author's local checkout but not in CI or other contributors' working trees.
- **Reference-style definition pointing nowhere**: Defining `[setup]: docs/old-setup.md` at the bottom of the file after the target was deleted. The definition is still parsed even when no inline `[setup]` consumes it.

## Pass / Fail

### Pass

~~~~markdown
# Project Setup

See [Getting Started](docs/getting-started.md) for the install steps and
the [testing rules](.claude/rules/testing.md) for the QA gate.

[setup]: docs/getting-started.md
~~~~

### Fail

~~~~markdown
# Project Setup

See [Getting Started](docs/old-setup.md) for the install steps and
the [testing rules](.claude/rules/deleted-rule.md) for the QA gate.

[setup]: docs/old-setup.md
~~~~

## Limitations

Discovers `[text](path)` inline links and `[ref]: path` reference definitions, then resolves each target relative to the source file's directory. Skips URLs (`://`, `mailto:`), absolute paths (`/foo`), and anchor-only references (`#frag`). Does not validate that anchors exist within the target file, does not detect broken references inside fenced code blocks (lookups span the entire file content), and does not check external URLs for reachability.
Loading
Loading