Skip to content

Commit d13a813

Browse files
ArchieIndianclaude
andauthored
Claude/festive moore (#24)
* Add skill-doctor: diagnose silent skill discovery failures Runs 6 diagnostic checks per skill (YAML parse, required fields, path conventions, cron format, stateful coherence, schema validity). Exits 1 when FAILs are present — suitable as a post-install gate in install.sh. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Add installed-skill-auditor: weekly post-install security audit Detects INJECTION, CREDENTIAL, EXFILTRATION, DRIFT, and ORPHAN issues in all installed skills. Maintains content baselines for drift detection. Cron: Mondays 9am. Exits 1 on CRITICAL findings. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Add skill-trigger-tester: validate description trigger quality before publish Scores a skill's description against should-fire/should-not-fire prompt sets. Computes precision, recall, F1, and assigns a grade A–F. Exits 1 on grade C or lower, suitable as a pre-publish gate. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Add skill-loadout-manager: named skill profiles to manage context bloat Defines and switches between curated skill subsets (loadouts). Ships 4 presets (minimal, coding, research, ops) and estimates token savings per loadout vs. all-skills-active mode. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Add skill-compatibility-checker: detect version/feature incompatibilities Reads requires_openclaw + requires_features frontmatter fields and compares against detected (or overridden) OpenClaw version. Ships feature registry with 5 runtime capabilities and their introduction versions. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Add skill-conflict-detector: detect name shadowing and description overlap (#21) Detects NAME_SHADOW (CRITICAL), EXACT_DUPLICATE (CRITICAL), HIGH_OVERLAP (HIGH), and MEDIUM_OVERLAP (MEDIUM) conflicts between installed skills using Jaccard similarity on description tokens. Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> * Add heartbeat-governor: per-skill execution budgets for cron skills (#22) Tracks 30-day rolling spend and wall-clock time per scheduled skill. Auto-pauses skills that exceed monthly/per-run budgets. Cron: every hour. Supports manual pause/resume and per-skill budget overrides. Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> * Add skill-portability-checker: validate OS/binary dependencies in scripts (#23) Detects OS_SPECIFIC_CALL, MISSING_BINARY, BREW_ONLY, PYTHON_IMPORT, and HARDCODED_PATH issues in companion scripts. Cross-checks against os_filter: frontmatter field. No external dependencies. Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent b0535d1 commit d13a813

26 files changed

Lines changed: 3981 additions & 0 deletions

File tree

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
---
2+
name: skill-conflict-detector
3+
version: "1.0"
4+
category: core
5+
description: Detects skill name shadowing and description-overlap conflicts that cause OpenClaw to trigger the wrong skill or silently ignore one when two skills compete for the same intent.
6+
---
7+
8+
# Skill Conflict Detector
9+
10+
## What it does
11+
12+
Two types of conflict cause skills to misbehave silently:
13+
14+
**1. Name shadowing** — Two installed skills have the same `name:` field. OpenClaw loads the last one lexicographically; the other silently disappears. No warning.
15+
16+
**2. Description overlap** — Two skills' descriptions are so semantically similar that OpenClaw can't reliably distinguish them. The wrong skill fires. You think one skill is broken; actually the other is intercepting it.
17+
18+
Skill Conflict Detector scans all installed skills for both types and reports them with overlap scores and resolution suggestions.
19+
20+
## When to invoke
21+
22+
- After installing a new skill from ClawHub
23+
- When a skill fires inconsistently or triggers on unexpected prompts
24+
- Before publishing a new skill (ensure it doesn't shadow an existing one)
25+
- As part of `install.sh` post-install validation
26+
27+
## Conflict types
28+
29+
| Type | Severity | Effect |
30+
|---|---|---|
31+
| NAME_SHADOW | CRITICAL | One skill completely hidden |
32+
| EXACT_DUPLICATE | CRITICAL | Identical description — both fire or neither does |
33+
| HIGH_OVERLAP | HIGH | >75% semantic similarity — unreliable trigger routing |
34+
| MEDIUM_OVERLAP | MEDIUM | 50–75% similarity — possible confusion |
35+
36+
## Output
37+
38+
```
39+
Skill Conflict Report — 32 skills
40+
────────────────────────────────────────────────
41+
0 CRITICAL | 1 HIGH | 0 MEDIUM
42+
43+
HIGH skill-vetting ↔ installed-skill-auditor overlap: 0.81
44+
Both describe "scanning skills for security issues"
45+
Suggestion: Differentiate — skill-vetting is pre-install,
46+
installed-skill-auditor is post-install ongoing audit.
47+
```
48+
49+
## How to use
50+
51+
```bash
52+
python3 detect.py --scan # Full conflict scan
53+
python3 detect.py --scan --skill my-skill # Check one skill vs all others
54+
python3 detect.py --scan --threshold 0.6 # Custom similarity threshold
55+
python3 detect.py --names # Check name shadowing only
56+
python3 detect.py --format json
57+
```
58+
59+
## Procedure
60+
61+
**Step 1 — Run the scan**
62+
63+
```bash
64+
python3 detect.py --scan
65+
```
66+
67+
**Step 2 — Resolve CRITICAL conflicts first**
68+
69+
NAME_SHADOW: Rename one skill's `name:` field and its directory. Run `bash scripts/validate-skills.sh` to confirm.
70+
71+
EXACT_DUPLICATE: One skill is redundant. Remove or differentiate it.
72+
73+
**Step 3 — Assess HIGH_OVERLAP pairs**
74+
75+
Read both descriptions. Ask: could a user's natural-language request unambiguously route to one and not the other? If no, differentiate. Common fix: add the scope or timing to the description (e.g., "before install" vs. "after install").
76+
77+
**Step 4 — Accept or suppress MEDIUM_OVERLAP**
78+
79+
Medium overlaps are informational. If the two skills serve genuinely different contexts and users would naturally phrase requests differently, they can coexist. Document why in the skill's SKILL.md if it's non-obvious.
80+
81+
## Similarity model
82+
83+
Token-overlap Jaccard similarity between description strings after stop-word removal. Fast and deterministic — no external dependencies.
84+
85+
Threshold defaults: HIGH ≥ 0.75, MEDIUM ≥ 0.50.
Lines changed: 293 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,293 @@
1+
#!/usr/bin/env python3
2+
"""
3+
Skill Conflict Detector for openclaw-superpowers.
4+
5+
Detects name shadowing and description-overlap conflicts between
6+
installed skills that cause silent trigger routing failures.
7+
8+
Usage:
9+
python3 detect.py --scan
10+
python3 detect.py --scan --skill my-skill
11+
python3 detect.py --scan --threshold 0.6
12+
python3 detect.py --names # Name shadowing only
13+
python3 detect.py --format json
14+
"""
15+
16+
import argparse
17+
import json
18+
import os
19+
import re
20+
import sys
21+
from pathlib import Path
22+
23+
try:
24+
import yaml
25+
HAS_YAML = True
26+
except ImportError:
27+
HAS_YAML = False
28+
29+
SUPERPOWERS_DIR = Path(os.environ.get(
30+
"SUPERPOWERS_DIR",
31+
Path.home() / ".openclaw" / "extensions" / "superpowers"
32+
))
33+
SKILLS_DIRS = [
34+
SUPERPOWERS_DIR / "skills" / "core",
35+
SUPERPOWERS_DIR / "skills" / "openclaw-native",
36+
SUPERPOWERS_DIR / "skills" / "community",
37+
]
38+
39+
DEFAULT_HIGH_THRESHOLD = 0.75
40+
DEFAULT_MEDIUM_THRESHOLD = 0.50
41+
42+
_STOPWORDS = {
43+
"a", "an", "the", "and", "or", "but", "in", "on", "at", "to", "for",
44+
"of", "with", "by", "from", "is", "are", "was", "were", "be", "been",
45+
"it", "its", "this", "that", "so", "not", "no", "all", "any", "each",
46+
"more", "most", "has", "have", "had", "do", "does", "did", "will",
47+
"would", "could", "should", "may", "can", "which", "when", "where",
48+
"how", "what", "who", "i", "you", "we", "they", "he", "she",
49+
}
50+
51+
52+
# ── Frontmatter parser ────────────────────────────────────────────────────────
53+
54+
def parse_frontmatter(skill_md: Path) -> dict:
55+
try:
56+
text = skill_md.read_text()
57+
lines = text.splitlines()
58+
if not lines or lines[0].strip() != "---":
59+
return {}
60+
end = None
61+
for i, line in enumerate(lines[1:], 1):
62+
if line.strip() == "---":
63+
end = i
64+
break
65+
if end is None:
66+
return {}
67+
fm_text = "\n".join(lines[1:end])
68+
if HAS_YAML:
69+
return yaml.safe_load(fm_text) or {}
70+
fields = {}
71+
for line in fm_text.splitlines():
72+
if ":" in line and not line.startswith(" "):
73+
k, _, v = line.partition(":")
74+
fields[k.strip()] = v.strip().strip('"').strip("'")
75+
return fields
76+
except Exception:
77+
return {}
78+
79+
80+
# ── Tokeniser + similarity ────────────────────────────────────────────────────
81+
82+
def tokenise(text: str) -> set[str]:
83+
tokens = re.findall(r"[a-z0-9]+", text.lower())
84+
return {t for t in tokens if t not in _STOPWORDS and len(t) > 2}
85+
86+
87+
def jaccard(a: set, b: set) -> float:
88+
if not a and not b:
89+
return 1.0
90+
inter = len(a & b)
91+
union = len(a | b)
92+
return inter / union if union > 0 else 0.0
93+
94+
95+
# ── Skill loader ──────────────────────────────────────────────────────────────
96+
97+
def load_all_skills() -> list[dict]:
98+
skills = []
99+
for skills_root in SKILLS_DIRS:
100+
if not skills_root.exists():
101+
continue
102+
for skill_dir in sorted(skills_root.iterdir()):
103+
if not skill_dir.is_dir():
104+
continue
105+
skill_md = skill_dir / "SKILL.md"
106+
if not skill_md.exists():
107+
continue
108+
fm = parse_frontmatter(skill_md)
109+
skills.append({
110+
"dir_name": skill_dir.name,
111+
"name": fm.get("name", skill_dir.name),
112+
"description": fm.get("description", ""),
113+
"path": str(skill_md),
114+
})
115+
return skills
116+
117+
118+
# ── Conflict detection ────────────────────────────────────────────────────────
119+
120+
def detect_conflicts(skills: list[dict], high_threshold: float,
121+
medium_threshold: float,
122+
single_skill: str = None) -> list[dict]:
123+
conflicts = []
124+
125+
# Name shadowing: same name field, different directories
126+
by_name: dict = {}
127+
for s in skills:
128+
by_name.setdefault(s["name"], []).append(s)
129+
130+
for name, group in by_name.items():
131+
if len(group) > 1:
132+
for i in range(len(group)):
133+
for j in range(i + 1, len(group)):
134+
a, b = group[i], group[j]
135+
if single_skill and single_skill not in (a["dir_name"], b["dir_name"]):
136+
continue
137+
conflicts.append({
138+
"type": "NAME_SHADOW",
139+
"severity": "CRITICAL",
140+
"skill_a": a["dir_name"],
141+
"skill_b": b["dir_name"],
142+
"overlap_score": 1.0,
143+
"detail": f"Both have `name: {name}` — one will be hidden",
144+
"suggestion": f"Rename one skill's `name:` field and its directory.",
145+
})
146+
147+
# Description overlap
148+
for i in range(len(skills)):
149+
for j in range(i + 1, len(skills)):
150+
a, b = skills[i], skills[j]
151+
if single_skill and single_skill not in (a["dir_name"], b["dir_name"]):
152+
continue
153+
154+
ta = tokenise(a["description"])
155+
tb = tokenise(b["description"])
156+
157+
if not ta or not tb:
158+
continue
159+
160+
score = jaccard(ta, tb)
161+
162+
if score >= high_threshold:
163+
# Check for exact duplicate
164+
severity = "CRITICAL" if a["description"] == b["description"] else "HIGH"
165+
ctype = "EXACT_DUPLICATE" if severity == "CRITICAL" else "HIGH_OVERLAP"
166+
common = ta & tb
167+
conflicts.append({
168+
"type": ctype,
169+
"severity": severity,
170+
"skill_a": a["dir_name"],
171+
"skill_b": b["dir_name"],
172+
"overlap_score": round(score, 3),
173+
"detail": (
174+
f"Descriptions share key terms: "
175+
+ ", ".join(f'"{t}"' for t in sorted(common)[:5])
176+
),
177+
"suggestion": (
178+
"Differentiate descriptions — add scope, timing, or "
179+
"context that distinguishes when each skill fires."
180+
),
181+
})
182+
elif score >= medium_threshold:
183+
common = ta & tb
184+
conflicts.append({
185+
"type": "MEDIUM_OVERLAP",
186+
"severity": "MEDIUM",
187+
"skill_a": a["dir_name"],
188+
"skill_b": b["dir_name"],
189+
"overlap_score": round(score, 3),
190+
"detail": (
191+
"Moderate description overlap — "
192+
+ ", ".join(f'"{t}"' for t in sorted(common)[:4])
193+
),
194+
"suggestion": (
195+
"Acceptable if use-cases are clearly distinct. "
196+
"Consider adding differentiating context to each description."
197+
),
198+
})
199+
200+
return conflicts
201+
202+
203+
# ── Output ────────────────────────────────────────────────────────────────────
204+
205+
def print_report(conflicts: list, skills_count: int, fmt: str) -> None:
206+
criticals = [c for c in conflicts if c["severity"] == "CRITICAL"]
207+
highs = [c for c in conflicts if c["severity"] == "HIGH"]
208+
mediums = [c for c in conflicts if c["severity"] == "MEDIUM"]
209+
210+
if fmt == "json":
211+
print(json.dumps({
212+
"skills_scanned": skills_count,
213+
"critical_count": len(criticals),
214+
"high_count": len(highs),
215+
"medium_count": len(mediums),
216+
"conflicts": conflicts,
217+
}, indent=2))
218+
return
219+
220+
print(f"\nSkill Conflict Report — {skills_count} skills")
221+
print("─" * 50)
222+
print(f" {len(criticals)} CRITICAL | {len(highs)} HIGH | {len(mediums)} MEDIUM")
223+
print()
224+
225+
if not conflicts:
226+
print(" ✓ No conflicts detected.")
227+
else:
228+
for c in conflicts:
229+
icon = "✗" if c["severity"] in ("CRITICAL",) else (
230+
"!" if c["severity"] == "HIGH" else "⚠"
231+
)
232+
score_str = f" overlap: {c['overlap_score']:.2f}" if c["type"] != "NAME_SHADOW" else ""
233+
print(f" {icon} {c['severity']:8s} {c['skill_a']}{c['skill_b']}"
234+
f"{score_str}")
235+
print(f" {c['detail']}")
236+
print(f" → {c['suggestion']}")
237+
print()
238+
239+
240+
# ── Commands ──────────────────────────────────────────────────────────────────
241+
242+
def cmd_scan(high_threshold: float, medium_threshold: float,
243+
single_skill: str, fmt: str) -> None:
244+
skills = load_all_skills()
245+
conflicts = detect_conflicts(skills, high_threshold, medium_threshold, single_skill)
246+
print_report(conflicts, len(skills), fmt)
247+
critical_count = sum(1 for c in conflicts if c["severity"] == "CRITICAL")
248+
sys.exit(1 if critical_count > 0 else 0)
249+
250+
251+
def cmd_names(fmt: str) -> None:
252+
skills = load_all_skills()
253+
conflicts = detect_conflicts(skills, high_threshold=2.0, medium_threshold=2.0)
254+
name_conflicts = [c for c in conflicts if c["type"] == "NAME_SHADOW"]
255+
if fmt == "json":
256+
print(json.dumps(name_conflicts, indent=2))
257+
else:
258+
if not name_conflicts:
259+
print("✓ No name shadowing detected.")
260+
else:
261+
for c in name_conflicts:
262+
print(f"✗ SHADOW: {c['skill_a']}{c['skill_b']} {c['detail']}")
263+
sys.exit(1 if name_conflicts else 0)
264+
265+
266+
# ── Main ──────────────────────────────────────────────────────────────────────
267+
268+
def main():
269+
parser = argparse.ArgumentParser(description="Skill Conflict Detector")
270+
group = parser.add_mutually_exclusive_group(required=True)
271+
group.add_argument("--scan", action="store_true")
272+
group.add_argument("--names", action="store_true",
273+
help="Check name shadowing only")
274+
parser.add_argument("--skill", metavar="NAME",
275+
help="Check one skill against all others")
276+
parser.add_argument("--threshold", type=float, default=DEFAULT_HIGH_THRESHOLD,
277+
help=f"HIGH similarity threshold (default: {DEFAULT_HIGH_THRESHOLD})")
278+
parser.add_argument("--format", choices=["text", "json"], default="text")
279+
args = parser.parse_args()
280+
281+
if args.names:
282+
cmd_names(args.format)
283+
elif args.scan:
284+
cmd_scan(
285+
high_threshold=args.threshold,
286+
medium_threshold=DEFAULT_MEDIUM_THRESHOLD,
287+
single_skill=args.skill,
288+
fmt=args.format,
289+
)
290+
291+
292+
if __name__ == "__main__":
293+
main()

0 commit comments

Comments
 (0)