You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
feat(security): release-time malicious-content scan of the public skill surface (board #2)
New scripts/scan-skill-security.js (npm run security:scan): the complementary
signal to the privacy gate. Privacy (privacy-patterns.js) stops the author
private data leaking OUT at export; this scans published marketplace/skills
bodies for execution/exfiltration patterns (curl|bash, base64-decode-and-eval,
reverse shells, fork bombs, broad rm -rf, curl data-exfil, eval-of-fetched)
and over-broad allowed-tools (P1 */all, P2 Bash(*), P3 bare shell;
Bash(git:*) is clean).
Advisory by default (exit 0 -- teaching skills legitimately show shell patterns
as anti-examples); --strict hard-gates. Wired into release:check (advisory,
release-time, NOT verify:system); unit test in test:unit so the logic is
blocking-tested. First run: 180 scanned, 37 P3 bare-Bash advisories, 0 exec
matches. Decided as a scan gate, not a 5th verdict.
Also fixes a stale privacy-patterns.js comment naming a non-existent
.github/workflows/privacy-scan.yml.
Verified: 0.12s scan (no ReDoS); test-scan-skill-security green; package.json
valid; release:check chain unbroken.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
npm run release:check # release-readiness gate — the corpus gates verify:system OMITS (manifest:validate, routing-eval, export:verify-skill-md, marketplace:verify)
680
+
npm run release:check # release-readiness gate — the corpus gates verify:system OMITS (manifest:validate, routing-eval, export:verify-skill-md, marketplace:verify) + the advisory security:scan
681
+
npm run security:scan # release-time MALICIOUS-CONTENT scan of the public surface (advisory — exit 0; --strict to hard-gate)
681
682
```
682
683
683
684
**Which to run depends on your work mode.**`npm run verify:system` is the gate for **SYSTEM work** (schema, scripts, protocol docs, fixtures): it excludes the corpus-completeness gates (`lint`, `manifest:fresh`, `manifest:validate`, `marketplace:verify`, `status:check`, `test:marketplace-corpus`, `routing-eval`, `export:verify-skill-md`) that go red purely because individual `skills/skills/**/SKILL.md` files have not yet been migrated to the current contract — those live in `npm run verify:corpus` (per SKI-319, so a parallel CONTENT session's corpus drift can never redden the SYSTEM gate). A SYSTEM change is shippable when `verify:system` is green. `npm run verify` (the full gate) only goes green once the corpus has been migrated through the audit loop (CONTENT work) — do **not** relax the schema or skip a gate to force the full `verify` green while CONTENT migration is paused.
@@ -686,6 +687,8 @@ npm run release:check # release-readiness gate — the corpus gates verify:sy
686
687
687
688
> **The superset-compat SHAPE gate IS in `verify:system` (migration-insensitive — note the distinction from the omitted `marketplace:verify`).** `npm run marketplace:superset` (`verify-skill-md-export.js --plain marketplace/skills`) validates that every exported `marketplace/skills/**/SKILL.md` carries only the allowlisted Agent-Skills top-level keys (`name`, `description`, `license`, `compatibility`, `metadata`, `allowed-tools`, `paths`), with all Skill Graph extension fields nested under `metadata:`. This is the SHAPE half of `marketplace:verify` **without** the freshness `--check` half: the exporter always produces superset-compatible frontmatter regardless of a skill's migration state, so this gate reddens ONLY on an exporter SYSTEM bug (a future field shadowing a standard Agent-Skills name) — never on corpus-migration backlog. That migration-insensitivity is precisely why it is a blocking SYSTEM gate while `marketplace:verify` (freshness-coupled) stays a corpus gate. (Added 2026-06-10 — board-meeting concern #3: "superset-compat claimed, not CI-enforced." The check existed in the non-blocking informational `verify` job; this makes it blocking.)
688
689
690
+
> **Two distinct content scans — do not conflate them.** (1) The **privacy gate** (`scripts/lib/privacy-patterns.js`, consumed by `export-marketplace-skills.js`) stops the author's PRIVATE data (Sales Hub paths, secrets, PII, internal DB surfaces) from leaking OUT to the public release — it **fails closed** at export time. (2) The **security scan** (`scripts/scan-skill-security.js`, `npm run security:scan`) is the complementary MALICIOUS-CONTENT signal: it scans the published `marketplace/skills` bodies for execution/exfiltration patterns (`curl | bash`, base64-decode-and-eval, reverse shells, fork bombs, broad destructive `rm -rf`) and over-broad `allowed-tools` (P1 `*`/`all` → P2 `Bash(*)` → P3 bare unrestricted shell). It is **advisory by default** (exit 0; these are teaching skills that legitimately show shell patterns as anti-examples) and runs in `release:check`; `--strict` (`npm run security:scan:strict`) makes any finding a failing gate. (Added 2026-06-10 — board-meeting concern #2: "no security/integrity signal." Scope was decided as a release-time scan gate, NOT a 5th Audit Status verdict.)
691
+
689
692
### Audit-evidence consistency gate (now in `npm run verify`)
690
693
691
694
The audit-evidence honesty gate is **wired into `npm run verify`** as of 2026-05-30. It was previously excluded on the rationale that historical verdict records claiming a graded verdict without their artifact would force `verify` red and block unrelated SYSTEM work. That rationale is obsolete: the stale records that caused the reds were run-records for skills no longer in the library (renamed/merged/deleted), so there was no SKILL.md to downgrade and no artifact to author — they were junk. They were deleted (workspace commit `73f9e0f` — in the `~/Development` workspace repo, NOT skill-graph; resolve via `git -C ~/Development show 73f9e0f`), the gate went green, and it is now part of the full gate so the evidence-honesty check is no longer institutionally excluded from "green."
Copy file name to clipboardExpand all lines: CHANGELOG.md
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -28,6 +28,8 @@ Skill Graph versions describe the contract at each checkpoint in the git history
28
28
29
29
### Added
30
30
31
+
- **Release-time MALICIOUS-CONTENT security scan of the public skill surface (board concern #2, 2026-06-10T17:17Z).** New `scripts/scan-skill-security.js` (`npm run security:scan`) — the complementary signal to the existing privacy gate. Where `scripts/lib/privacy-patterns.js` stops the author's PRIVATE data leaking OUT (export-time, fails closed), this scans the published `marketplace/skills` bodies for execution/exfiltration patterns (`curl | bash`, base64-decode-and-eval, reverse shells via `/dev/tcp`/`nc -e`, fork bombs, broad destructive `rm -rf` on `/`/`~`/`$HOME`, curl data-exfil to external URLs, eval-of-fetched-content) and over-broad `allowed-tools` with an honest severity gradient (P1 `*`/`all` → P2 whole-argument `Bash(*)` → P3 bare unrestricted shell; `Bash(git:*)` and other prefix-restricted scopes are CLEAN). **Advisory by default** (exit 0 — these are teaching skills that legitimately show shell patterns as anti-examples, so a finding is a human-review prompt, not an auto-block); `--strict` (`npm run security:scan:strict`) makes any finding a failing gate. Wired into `release:check` (advisory, release-time — NOT `verify:system`); the unit test `test-scan-skill-security.js` is in `test:unit` (so the scanner LOGIC is blocking-tested). First run: 180 public skills scanned, 37 findings — all P3 "consider an `allowed-tools` allowlist" on conventional bare `Bash`; zero execution/exfil matches. Scope was decided (per the board) as a release-time scan gate, NOT a 5th Audit Status verdict (a verdict would cascade an earned-not-bumped schema migration across the corpus — too heavy to gate a release). Also corrected a stale comment in `privacy-patterns.js` that named a non-existent `.github/workflows/privacy-scan.yml` CI workflow. Verified: scanner runs in 0.12s (no ReDoS); unit test green; `release:check` chain unbroken (advisory exit 0). SYSTEM-only; no skill content touched.
32
+
31
33
- **Superset-compat marketplace SHAPE gate is now BLOCKING in `verify:system` (board concern #3, 2026-06-10T17:17Z).** The exported-frontmatter superset-compatibility check (`scripts/verify-skill-md-export.js` — every `marketplace/skills/**/SKILL.md` may carry only the allowlisted Agent-Skills top-level keys `name`/`description`/`license`/`compatibility`/`metadata`/`allowed-tools`/`paths`, all Skill Graph extensions nested under `metadata:`) existed but ran ONLY inside `verify` — the `continue-on-error: true` informational CI job (`.github/workflows/skill-graph-lint.yml`). So a future field shadowing a standard Agent-Skills name would NOT have failed CI. Added `npm run marketplace:superset` (`verify-skill-md-export.js --plain marketplace/skills`) — the SHAPE half of `marketplace:verify` WITHOUT its migration-sensitive freshness `--check` — and wired it into `verify:system` (the blocking CI gate). Migration-insensitive: the exporter always emits superset-compatible frontmatter regardless of a skill's migration state, so the gate reddens ONLY on an exporter SYSTEM bug, never on corpus-migration backlog — which is why it correctly belongs in the SYSTEM tier, not the corpus tier (respects the SKI-319 SYSTEM/corpus separation; documented in `AGENTS.md § Validation Commands`). The existing `test-marketplace-export.js` unit assertion (`validateExportedFrontmatter(...).errors.length === 0` on the a11y fixture) already guarded the exporter path for one skill; this adds the corpus-wide blocking guarantee. Verified: `marketplace:superset` → 180/180 PASS on the current surface; `validateExportedFrontmatter` flags injected shadow keys (`version`, `subject` at top level) as errors. SYSTEM-only; no skill content touched.
32
34
33
35
- **OpenCode bash-tool timeout documented + mitigated in the OpenCode panel supervisor (SKI-378, 2026-06-10T13:25Z).** Research (OpenCode 1.16.2 + GitHub issues): the bash tool's DEFAULT per-call timeout is **120000ms (2 min)**, raised via the env var `OPENCODE_EXPERIMENTAL_BASH_DEFAULT_TIMEOUT_MS` (no `opencode.json` key exists for it in 1.16.2 — the requested `tool.timeout`/`timeout_ms` config is unimplemented). Since a panel skill runs ~68 min, the supervisor's "one driver call per skill" model was silently exposed to a 2-min kill. Fix: the supervisor prompt's RUN invocation now prefixes `OPENCODE_EXPERIMENTAL_BASH_DEFAULT_TIMEOUT_MS=5400000` (90 min, matching the driver's own `--timeout 5400`), and documents the verified default + the **unverified** secondary risk (opencode#25509: an AI-SDK `streamText` `stepMs=120000` hard cap may wrap the entire tool step regardless of the env var) with an empirical confirmation command and the background-with-status-file fallback if that cap still bites. Doc-only; SYSTEM-only; no skill content touched. Sources: opencode#25913 (default + env var; fixed by PR #28998), opencode#25509 (streamText stepMs hard cap, v1.14.31).
"verify": "npm run lint && npm run lint:template && npm run protocol:check && npm run docs:links && npm run docs:drift && npm run counts:check && npm run routing-config:check && npm run mirror:freeze && npm run charter:parity && npm run stability:check && npm run manifest:validate && npm run routing-eval && npm run export:verify-skill-md && npm run marketplace:verify && npm run status:check && npm run audit-manifest:check && npm run application-evals:check && npm run models:check && npm run overlap && npm run test:unit",
92
94
"verify:system": "node scripts/check-schema-constants.js && npm run protocol:check && npm run docs:links && npm run docs:drift && npm run counts:check && npm run routing-config:check && npm run mirror:freeze && npm run charter:parity && npm run stability:check && npm run lint:template && node scripts/skill-lint.js --path examples/fixture-skills && npm run application-evals:check && npm run models:check && npm run overlap && npm run marketplace:superset && npm run test:unit",
93
95
"verify:corpus": "npm run manifest:fresh && npm run manifest:validate && npm run marketplace:verify && npm run status:check && npm run test:marketplace-corpus && npm run application-evals:check:strict",
"release:check": "npm run manifest:fresh && npm run manifest:validate && npm run routing-eval && npm run export:verify-skill-md && npm run marketplace:verify && npm run status:check"
97
+
"release:check": "npm run manifest:fresh && npm run manifest:validate && npm run routing-eval && npm run export:verify-skill-md && npm run marketplace:verify && npm run status:check && npm run security:scan"
0 commit comments