Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ Add supporting material only as the skill needs it, following existing precedent

| Touching | Run |
|---|---|
| Any skill's `SKILL.md`/`README.md` prose | No dedicated linter for this tree — `node scripts/check-notations.mjs`'s link check is scoped to `notations/**/*.md` only, so check relative links by hand. |
| Any skill's `SKILL.md`/`README.md` prose | `node scripts/check-notations.mjs` and `node --test scripts/check-notations.test.mjs` — shared links, example IDs and layer trees, including installed onboarding guide paths and fixture-root links. |
| `onboard`'s family-selection matrix or `templates/` | `node scripts/check-skill-cheatsheet.mjs` |
| `onboard`'s scaffold logic | `python transitrix/skills/onboard/tests/test_skill_integrity.py` |
| `knowledge-store`'s linter or pattern doc | the test(s) under `transitrix/skills/knowledge-store/tests/` |
Expand Down
113 changes: 75 additions & 38 deletions scripts/check-notations.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
// Catches `.bpmn.yaml`-style drift and misfiled examples.
// E2 header — each example carries a top-level `notation: <short>` whose
// value matches its file extension.
// L1 links — every relative Markdown link in notations/**/*.md resolves to
// L1 links — relative Markdown links in notations/, method/ and skills/ resolve to
// an existing file (anchors and external URLs are skipped).
// T1 document sources — every `.ttrs` file is named <basename>.<kind>.ttrs,
// and no file ends `.trs` (the near-miss: one keystroke away, a different
Expand Down Expand Up @@ -56,16 +56,16 @@
// ID1 example-ID grammar — every ID-shaped token (a candidate whose TYPE
// prefix is registered in IDS_AND_REFERENCES.md §3) found in a fenced
// code block or a backtick span under method/**/*.md and
// notations/**/*.md satisfies §1's grammar (CAPABILITY's V/H address
// excepted, §2). A token that is itself a known rule code
// notations/**/*.md and transitrix/skills/**/*.md satisfies §1's
// grammar (CAPABILITY's V/H address excepted, §2). A token that is itself a known rule code
// (vocabulary.yaml rule_codes / deferred.rule_codes — several rule-code
// prefixes collide with a registered TYPE, e.g. ACTION-005, TERM-002)
// is not an element ID and is excluded. IDS_AND_REFERENCES.md and
// CONVENTIONS.md are excluded from the scan — both carry a deliberate
// ✓/✗ comparison table documenting invalid forms as negative examples.
// LAYER1 layer enumeration (extends VOC1's reach) — a contiguous group of
// three or more distinct `NN_<word>/` layer-folder tokens under
// method/**/*.md or notations/**/*.md (a directory tree, a table) is
// method/, notations/ or transitrix/skills/ Markdown (a directory tree) is
// read as an attempt to enumerate the full layer set and must equal it
// exactly — no fewer, no extra. A single incidental folder citation
// (one element spec naming its own home folder) is not a "list" and is
Expand Down Expand Up @@ -344,36 +344,68 @@ async function checkCatalogueBoundary(failures, warnings) {
}
}

// L1: relative markdown links in notations/**/*.md and method/**/*.md must resolve.
async function checkLinks(failures) {
const files = (await walk(NOTATIONS_DIR, '.md')).concat(await walk(METHOD_DIR, '.md'));
// Inline links only: no newline in the link text or the target, so we never
// splice an unrelated `[` and `](…)` across prose into a phantom link.
// Shared instruction inventory: include supporting prompts, templates and fixture docs.
export async function collectInstructionMarkdown(root = REPO_ROOT) {
return (await walk(join(root, 'notations'), '.md')).concat(
await walk(join(root, 'method'), '.md'),
await walk(join(root, 'transitrix', 'skills'), '.md'));
}

// These guides are copied to the adopter root; Copilot is installed below .github.
// Resolve in that destination namespace, then require the backing source to exist.
// Other templates keep their source-relative links; no template directory is exempt.
const ONBOARD_GUIDES = new Map([
['AGENTS.md', 'AGENTS.md'], ['ANALYST.md', 'ANALYST.md'],
['VALIDATOR.md', 'VALIDATOR.md'], ['INGEST.md', 'INGEST.md'],
['FINDINGS.md', 'FINDINGS.md'],
['copilot-instructions.md', '.github/copilot-instructions.md'],
]);

export function resolveInstructionLink(abs, target, root = REPO_ROOT) {
const rel = relative(root, abs).split('\\').join('/');
const prefix = 'transitrix/skills/onboard/templates/';
const source = rel.startsWith(prefix) ? rel.slice(prefix.length) : null;
if (ONBOARD_GUIDES.has(source)) {
const installed = posix.normalize(posix.join(posix.dirname(ONBOARD_GUIDES.get(source)), target));
for (const [file, destination] of ONBOARD_GUIDES) {
if (destination === installed) return join(root, prefix, file);
}
return null; // not an installed guide, even if a same-named source file exists
}
// Knowledge fixtures use site-root /knowledge links. Bind to the fixture's
// own root, never the machine root or a different fixture's knowledge tree.
const fixture = rel.match(/^(transitrix\/skills\/knowledge-store\/tests\/fixtures\/[^/]+)\//);
if (fixture && target.startsWith('/')) {
if (target.split('/').includes('..')) return null;
const fixtureRoot = resolve(root, fixture[1]);
const resolved = resolve(fixtureRoot, '.' + target);
return resolved.startsWith(fixtureRoot + '/') ? resolved : null;
}
return resolve(dirname(abs), target);
}

export function findInstructionLinkFailures(text, abs, root = REPO_ROOT) {
const failures = [];
const rel = relative(root, abs).split('\\').join('/');
const linkRe = /\[[^\]\n]*\]\(([^)\n]+)\)/g;
for (const abs of files) {
const rel = relPosix(abs);
const text = await readFile(abs, 'utf8');
let m;
while ((m = linkRe.exec(text)) !== null) {
let target = m[1].trim();
// Skip external, anchor-only, mailto, and template placeholders.
if (/^(https?:|mailto:|#|<)/.test(target)) continue;
target = target.split('#')[0]; // drop anchor
if (!target) continue; // was anchor-only
if (target.includes('<') || target.includes('>')) continue; // placeholder like <id>.yaml
// Only validate things that look like a local file/dir reference — a path
// segment (`/`) or a file extension. Bare words in parenthetical prose
// (`(factor)`, `(set_name)`) are not links.
if (!target.includes('/') && !/\.[a-z0-9]+$/i.test(target)) continue;
const resolved = resolve(dirname(abs), target);
if (!existsSync(resolved)) {
failures.push({
check: 'L1',
message: `${rel}: broken relative link → \`${m[1]}\` (resolves to ${relPosix(resolved)}, not found).`,
});
}
for (const m of text.matchAll(linkRe)) {
let target = m[1].trim();
if (/^(https?:|mailto:|#|<)/.test(target)) continue;
target = target.split('#')[0];
if (!target || target.includes('<') || target.includes('>')) continue;
if (!target.includes('/') && !/\.[a-z0-9]+$/i.test(target)) continue;
const resolved = resolveInstructionLink(abs, target, root);
if (!resolved || !existsSync(resolved)) {
failures.push({ check: 'L1', message: `${rel}: broken relative link → \`${m[1]}\` (destination not found).` });
}
}
return failures;
}

async function checkLinks(failures) {
for (const abs of await collectInstructionMarkdown()) {
failures.push(...findInstructionLinkFailures(await readFile(abs, 'utf8'), abs));
}
}

// V1: concrete methodology_version pins must equal the SoT.
Expand Down Expand Up @@ -1398,7 +1430,7 @@ const ID_TOKEN_RE = /\b[A-Z][A-Z0-9_]*(?:-[A-Za-z0-9_]+(?:\.[0-9]+)*)+\b/g;
// "table cell" — this repo backtick-quotes every ID it documents). Skips a
// match that is itself a placeholder by this repo's own conventions: wrapped
// in `<…>` (`<STEP-id>`, `<REQUIREMENT-HINT>`) or immediately followed by the
// `-…` family-prefix ellipsis (`NEED-VALIDATION-COVERAGE-…`). Returns
// `-…` / `-...` family-prefix ellipsis or a shell `-*` prefix pattern. Returns
// [{ token, line }] with 1-based line numbers.
export function findIdCandidates(text) {
const out = [];
Expand All @@ -1410,8 +1442,13 @@ export function findIdCandidates(text) {
const afterIdx = m.index + m[0].length;
const after = segment[afterIdx];
if (before === '<' && after === '>') continue; // <STEP-id>-style placeholder
if (after === '…' || segment.slice(afterIdx, afterIdx + 2) === '-…') continue; // family-prefix marker
out.push({ token: m[0], line });
if (after === '…' || /^(?:-…|-\.\.\.|-\*)/.test(segment.slice(afterIdx))) continue; // family-prefix marker
// Operational snapshots append a fetch date to a codex ID. Check the
// source ID, not the filename's date, only in the declared snapshot path.
const snapshot = segment.slice(0, m.index).endsWith('_intake/snapshots/')
&& /^\.(?:html|pdf|json|xml)\b/.test(segment.slice(afterIdx))
&& m[0].match(/^((?:REGULATION|STANDARD|POLICY|INTERNAL_STANDARD)-.+)-\d{4}-\d{2}-\d{2}$/);
out.push({ token: snapshot ? snapshot[1] : m[0], line });
}
};
for (let i = 0; i < lines.length; i++) {
Expand All @@ -1431,7 +1468,7 @@ export function findIdCandidates(text) {
return out;
}

async function checkIdGrammar(failures) {
export async function checkIdGrammar(failures, files = null) {
let registryText;
try {
registryText = await readFile(IDS_AND_REFERENCES_PATH, 'utf8');
Expand All @@ -1457,7 +1494,7 @@ async function checkIdGrammar(failures) {
// "no rule-code exclusions" rather than duplicating that failure here.
}

const files = (await walk(NOTATIONS_DIR, '.md')).concat(await walk(METHOD_DIR, '.md'));
files ??= await collectInstructionMarkdown();
const seen = new Set();
for (const abs of files) {
if (ID1_EXCLUDED_FILES.has(abs)) continue; // documents invalid forms on purpose
Expand Down Expand Up @@ -1534,9 +1571,9 @@ export function findLayerEnumerationGroups(text) {
return groups.filter(g => g.folders.size >= 3);
}

async function checkLayerEnumeration(failures) {
export async function checkLayerEnumeration(failures, files = null) {
const canonical = new Set(Object.values(LAYER_WORD_TO_FOLDER));
const files = (await walk(NOTATIONS_DIR, '.md')).concat(await walk(METHOD_DIR, '.md'));
files ??= await collectInstructionMarkdown();
for (const abs of files) {
const rel = relPosix(abs);
const text = await readFile(abs, 'utf8');
Expand Down
128 changes: 127 additions & 1 deletion scripts/check-notations.test.mjs
Original file line number Diff line number Diff line change
@@ -1,13 +1,20 @@
// Unit tests for the C1 class-derivation logic in check-notations.mjs.
// Run: node --test scripts/check-notations.test.mjs
//
// Exercises the pure functions only (no filesystem, no subprocess) — the
// Exercises pure functions and isolated instruction-path fixtures — the
// integration-level guarantee (the real repo's counts match its README) is
// covered by running scripts/check-notations.mjs itself in CI.

import { test } from 'node:test';
import assert from 'node:assert/strict';
import { mkdtempSync, mkdirSync, writeFileSync, readFileSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join, resolve } from 'node:path';
import {
collectInstructionMarkdown,
findInstructionLinkFailures,
checkIdGrammar,
checkLayerEnumeration,
deriveClassCounts,
parseStatedViewCounts,
findStandardIdentifierEmissions,
Expand Down Expand Up @@ -891,3 +898,122 @@ test('relation endpoint tables retain Project subtype and exclude driver-to-driv
assert.deepEqual(vocabulary.get('source_trace').excludedPairs, rows.get('source_trace').excludedPairs);
assert.throws(() => parseRelationsEnumTable(text.replace('ACTION(Project)', 'PRODUCT(Project)')), /unexpected subtype/);
});


// Maintainers exercise the same checker against installed instruction namespaces.
// Fixtures stay ephemeral; the published skill files are checked in place in CI.
test('skill instruction inventory includes protocols, prompts, templates and fixture docs', async () => {
const files = await collectInstructionMarkdown();
for (const name of [
'feedback/SKILL.md', 'knowledge-store/README.md',
'onboard/extraction/01_motivation.md', 'onboard/templates/AGENTS.md',
'knowledge-store/tests/fixtures/valid/knowledge/warehouse-finance-handoff.md',
]) assert.ok(files.includes(resolve('transitrix/skills', name)), name);
});

test('instruction links resolve source, installed and fixture roots and reject missing targets', () => {
const root = mkdtempSync(join(tmpdir(), 'instruction-links-'));
const put = (name, text = '') => {
const path = join(root, name);
mkdirSync(join(path, '..'), { recursive: true });
writeFileSync(path, text);
return path;
};
try {
const skill = put('transitrix/skills/knowledge-store/SKILL.md');
put('patterns/knowledge-store.md');
assert.equal(findInstructionLinkFailures('[gate](../../../patterns/knowledge-store.md#gate)', skill, root).length, 0);
assert.equal(findInstructionLinkFailures('[gate](../../../../patterns/knowledge-store.md)', skill, root).length, 1);
const agent = put('transitrix/skills/onboard/templates/AGENTS.md');
const copilot = put('transitrix/skills/onboard/templates/copilot-instructions.md');
assert.equal(findInstructionLinkFailures('[pointer](.github/copilot-instructions.md)', agent, root).length, 0);
assert.equal(findInstructionLinkFailures('[guide](../AGENTS.md)', copilot, root).length, 0);
assert.equal(findInstructionLinkFailures('[guide](../AGENT.md)', copilot, root).length, 1);
assert.equal(findInstructionLinkFailures('[pointer](copilot-instructions.md)', agent, root).length, 1);
rmSync(copilot);
assert.equal(findInstructionLinkFailures('[pointer](.github/copilot-instructions.md)', agent, root).length, 1);
const fixture = put('transitrix/skills/knowledge-store/tests/fixtures/valid/knowledge/a.md');
put('transitrix/skills/knowledge-store/tests/fixtures/valid/knowledge/b.md');
assert.equal(findInstructionLinkFailures('[object](/knowledge/b.md)', fixture, root).length, 0);
assert.equal(findInstructionLinkFailures('[object](/knowledge/missing.md)', fixture, root).length, 1);
const other = put('transitrix/skills/knowledge-store/tests/fixtures/other/knowledge/a.md');
assert.equal(findInstructionLinkFailures('[object](/knowledge/b.md)', other, root).length, 1);
assert.equal(findInstructionLinkFailures('[escape](/../valid/knowledge/b.md)', fixture, root).length, 1);
const raw = put('transitrix/skills/ingest/tests/README.md');
put('transitrix/skills/ingest/tests/fixtures/raw/INTERVIEW-sample.md');
assert.equal(findInstructionLinkFailures('[raw](fixtures/raw/INTERVIEW-sample.md)', raw, root).length, 0);
assert.equal(findInstructionLinkFailures('[raw](fixtures/raw/INTERVIEW-missing.md)', raw, root).length, 1);
} finally { rmSync(root, { recursive: true, force: true }); }
});

test('instruction IDs distinguish explicit placeholders, prose and snapshot source IDs', async () => {
const root = mkdtempSync(join(tmpdir(), 'instruction-ids-'));
const file = join(root, 'SKILL.md');
try {
writeFileSync(file, [
'`<GOAL-NNN>`', '`REGULATION-CFR-*`', '`INTERVIEW-cfo-...`',
'```yaml', 'description: PROCESS level extraction',
'id: CAPABILITY-V3.2', 'goal: GOAL-EXAMPLE-1', '```',
'`_intake/snapshots/REGULATION-GDPR-2016-1-2026-06-08.html`',
].join('\n'));
const good = [];
await checkIdGrammar(good, [file]);
assert.deepEqual(good, []);
writeFileSync(file, [
'`GOAL-NNN`', '`PROCESS-level`', '`CAPABILITY-ORDER_FULFIL-3`',
'`GOAL-EXAMPLE-01`',
'`_intake/snapshots/REGULATION-BAD-01-2026-06-08.html`',
].join('\n'));
const bad = [];
await checkIdGrammar(bad, [file]);
assert.equal(bad.length, 5);
assert.ok(bad.some(f => f.message.includes('REGULATION-BAD-01')));
} finally { rmSync(root, { recursive: true, force: true }); }
});

test('a broken real instruction and incomplete template tree still fail shared checks', async () => {
const root = mkdtempSync(join(tmpdir(), 'instruction-regression-'));
try {
const feedback = join(root, 'SKILL.md');
writeFileSync(feedback, readFileSync('transitrix/skills/feedback/SKILL.md', 'utf8')
.replaceAll('CAPABILITY-V3', 'CAPABILITY-ORDER_FULFIL-3'));
const ids = [];
await checkIdGrammar(ids, [feedback]);
assert.equal(ids.length, 1);
const knowledge = resolve('transitrix/skills/knowledge-store/SKILL.md');
const links = findInstructionLinkFailures(readFileSync(knowledge, 'utf8')
.replaceAll('../../../patterns/', '../../../../patterns/'), knowledge);
assert.equal(links.length, 5);
const template = join(root, 'AGENTS.md');
const text = readFileSync('transitrix/skills/onboard/templates/AGENTS.md', 'utf8');
writeFileSync(template, text);
const good = [];
await checkLayerEnumeration(good, [template]);
assert.deepEqual(good, []);
writeFileSync(template, text.split('\n').filter(l => !l.includes('└── 05_implementation/')).join('\n'));
const bad = [];
await checkLayerEnumeration(bad, [template]);
assert.equal(bad.length, 1);
assert.match(bad[0].message, /05_implementation/);
} finally { rmSync(root, { recursive: true, force: true }); }
});


test('actual onboarding guides resolve after copying to their documented destinations', async () => {
const root = mkdtempSync(join(tmpdir(), 'installed-guides-'));
const files = ['AGENTS.md', 'ANALYST.md', 'VALIDATOR.md', 'INGEST.md', 'FINDINGS.md', 'copilot-instructions.md'];
try {
mkdirSync(join(root, '.github'));
for (const file of files) {
const destination = file === 'copilot-instructions.md' ? '.github/' + file : file;
writeFileSync(join(root, destination), readFileSync('transitrix/skills/onboard/templates/' + file));
}
for (const file of files) {
const destination = join(root, file === 'copilot-instructions.md' ? '.github/' + file : file);
assert.deepEqual(findInstructionLinkFailures(readFileSync(destination, 'utf8'), destination, root), []);
}
rmSync(join(root, 'FINDINGS.md'));
const guide = join(root, 'AGENTS.md');
assert.ok(findInstructionLinkFailures(readFileSync(guide, 'utf8'), guide, root).length > 0);
} finally { rmSync(root, { recursive: true, force: true }); }
});
6 changes: 3 additions & 3 deletions transitrix/skills/feedback/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,8 +148,8 @@ correctly.
3. **Refuse to write** (state why, then return to Step 2 to rephrase) if the composed
text contains any of:
- a token matching the canonical ID grammar, `<TYPE>-[<middle>-]<INTEGER>`
(`notations/IDS_AND_REFERENCES.md` §1) — e.g. `GOAL-RETENTION-12`,
`CAPABILITY-ORDER_FULFIL-3` — this is an element ID, not a generic description;
(`notations/IDS_AND_REFERENCES.md` §1, with the capability V/H form in §2) — e.g. `GOAL-RETENTION-12`,
`CAPABILITY-V3` — this is an element ID, not a generic description;
- a path under `canon/`, `field/`, or `codex/`;
- an organisation name, a real person's name, or any other organisation-identifying
detail (the same real-names / "publish pattern, not client instance" rule applied
Expand All @@ -158,7 +158,7 @@ correctly.
cannot be stated without one of the above, ask the user to rephrase it more
generically. Loop until the text is clean or the user abandons the entry.

**Worked negative example** — an observation stated as *"our `CAPABILITY-ORDER_FULFIL-3`
**Worked negative example** — an observation stated as *"our `CAPABILITY-V3`
element has no relation kind to link it to `GOAL-RETENTION-12` at Acme Corp"* is
refused on all three grounds (two element IDs, one organisation name) and rewritten,
with the user, as *"no relation kind connects a CAPABILITY element to a GOAL
Expand Down
Loading
Loading