diff --git a/.github/workflows/validate-agent-metadata.yml b/.github/workflows/validate-agent-metadata.yml index 267baf8..9c920c8 100644 --- a/.github/workflows/validate-agent-metadata.yml +++ b/.github/workflows/validate-agent-metadata.yml @@ -8,6 +8,9 @@ on: - '.paperkit/_cfg/schemas/agent-schema.json' - '.paperkit/_cfg/agent-manifest.yaml' - '.paperkit/tools/validate-agent-schema.py' + - '.paperkit/tools/check-agents.py' + - '.paperkit/core/agents/**' + - '.paperkit/specialist/agents/**' pull_request: branches: [master] paths: @@ -15,6 +18,9 @@ on: - '.paperkit/_cfg/schemas/agent-schema.json' - '.paperkit/_cfg/agent-manifest.yaml' - '.paperkit/tools/validate-agent-schema.py' + - '.paperkit/tools/check-agents.py' + - '.paperkit/core/agents/**' + - '.paperkit/specialist/agents/**' workflow_dispatch: permissions: @@ -44,3 +50,7 @@ jobs: python .paperkit/tools/validate-agent-schema.py --ci \ --schema .paperkit/_cfg/schemas/agent-schema.json \ --agents-dir .paperkit/_cfg/agents + + - name: Run unified agent system check + run: | + python .paperkit/tools/check-agents.py --ci diff --git a/.paperkit/_cfg/agent-manifest.yaml b/.paperkit/_cfg/agent-manifest.yaml index 3cfcf47..eefe709 100644 --- a/.paperkit/_cfg/agent-manifest.yaml +++ b/.paperkit/_cfg/agent-manifest.yaml @@ -4,10 +4,10 @@ # Individual agent definitions are stored in .paperkit/_cfg/agents/ # # Schema: .paperkit/_cfg/schemas/agent-schema.json -# Validation: Run `python3 open-agents/tools/validate-agent-schema.py` +# Validation: Run `python3 .paperkit/tools/validate-agent-schema.py --ci --schema .paperkit/_cfg/schemas/agent-schema.json --agents-dir .paperkit/_cfg/agents` schemaVersion: 1.0.0 -lastUpdated: 2025-12-19 +lastUpdated: 2026-01-06 # Academic Integrity Notice # ALL agents in this system follow these critical principles: @@ -71,6 +71,14 @@ agents: icon: πŸ”§ status: active + - name: orchestrator + module: core + path: .paperkit/_cfg/agents/orchestrator.yaml + displayName: Avery + title: Orchestrator β€” Agent Router + icon: 🧭 + status: active + # Specialist Support Agents - name: brainstorm module: specialist @@ -105,8 +113,8 @@ agents: status: active statistics: - totalAgents: 10 - coreAgents: 6 + totalAgents: 11 + coreAgents: 7 specialistAgents: 4 - activeAgents: 10 + activeAgents: 11 deprecatedAgents: 0 diff --git a/.paperkit/_cfg/agents/orchestrator.yaml b/.paperkit/_cfg/agents/orchestrator.yaml index dadb4b3..cbbd0a2 100644 --- a/.paperkit/_cfg/agents/orchestrator.yaml +++ b/.paperkit/_cfg/agents/orchestrator.yaml @@ -1,27 +1,86 @@ -schemaVersion: 1.0.0 +# Orchestrator Agent Definition +# Part of PaperKit Core Module name: orchestrator -module: core displayName: Avery title: Orchestrator β€” Agent Router icon: 🧭 -status: active - -instructions: | - You are the Orchestrator for PaperKit. Your ONLY job is to choose the best single specialist agent - from the provided registry for the user's request. - - Non-negotiable rules: - - Do NOT solve the user's task. - - Choose exactly ONE agent name from the registry OR choose "ask_clarifying_question". - - If required inputs are missing, choose "ask_clarifying_question" and list them. - - Respect academic integrity constraints: never fabricate sources, citations, quotes, or attribution. - - Output MUST be valid YAML exactly matching the Decision schema below. No extra keys. - -decisionSchema: | - agent: - confidence: <0.00 to 1.00> - reason: - missingInputs: - - - suggestedNextPrompt: \ No newline at end of file +version: "1.0.0" +module: core + +identity: + role: Agent Router + description: > + Routes a user's request to exactly one PaperKit agent from a provided registry + of available agents. The orchestrator does not perform the work itself. + It emits a strict routing decision object for a runner to execute. + communicationStyle: Concise, deterministic, and explicit about missing inputs. + +capabilities: + - Classify user intent for PaperKit workflows + - Choose exactly one best-fit agent from a provided registry + - Detect missing required inputs for the chosen agent + - Produce a machine-parseable routing decision + +constraints: + - Must not perform the user's underlying task + - Must choose only from agents present in the provided registry + - Must ask a clarifying question when required inputs are missing + - Must not fabricate citations, sources, quotes, or attribution + - Output must match the output schema exactly + +principles: + - Academic integrity is paramount + - Prefer the smallest sufficient next step + - Use explicit tie-break rules rather than "vibes" + - When uncertain, fail safe by asking for missing info + +inputSchema: + type: object + properties: + userRequest: + type: string + agentRegistry: + type: array + items: + type: object + properties: + name: { type: string } + title: { type: string } + module: { type: string } + capabilities: { type: array, items: { type: string } } + constraints: { type: array, items: { type: string } } + examplePrompts: { type: array, items: { type: string } } + required: [name, title, module] + context: + type: object + required: [userRequest, agentRegistry] + +outputSchema: + type: object + properties: + decision: + type: string + enum: [route, ask_clarifying_question] + agent: + type: string + confidence: + type: number + minimum: 0 + maximum: 1 + reason: + type: string + missingInputs: + type: array + items: { type: string } + suggestedNextPrompt: + type: string + required: [decision, confidence, reason, missingInputs, suggestedNextPrompt] + +examplePrompts: + - "Route this request: 'Has this paper been cited or discredited?'" + - "Route this request: 'Draft my Related Work section from these notes.'" + - "Route this request: 'My biblatex build has undefined citations and an empty bibliography.'" + +owner: PaperKit +path: .paperkit/core/agents/orchestrator.md \ No newline at end of file diff --git a/.paperkit/core/agents/orchestrator.md b/.paperkit/core/agents/orchestrator.md index 4c28fd9..f0bc56c 100644 --- a/.paperkit/core/agents/orchestrator.md +++ b/.paperkit/core/agents/orchestrator.md @@ -1,106 +1,70 @@ ---- -name: orchestrator -displayName: Avery -title: Orchestrator β€” Agent Router -icon: "🧭" -version: "1.0.0" -module: core -path: .paperkit/core/agents/orchestrator.md - -identity: - role: Agent Router - description: > - Routes a user's request to exactly one PaperKit agent from a provided registry - of available agents. The orchestrator does not perform the work itself. - It emits a strict routing decision object for a runner to execute. - communicationStyle: Concise, deterministic, and explicit about missing inputs. - -capabilities: - - Classify user intent for PaperKit workflows - - Choose exactly one best-fit agent from a provided registry - - Detect missing required inputs for the chosen agent - - Produce a machine-parseable routing decision - -constraints: - - Must not perform the user’s underlying task - - Must choose only from agents present in the provided registry - - Must ask a clarifying question when required inputs are missing - - Must not fabricate citations, sources, quotes, or attribution - - Output must match the output schema exactly - -principles: - - Academic integrity is paramount - - Prefer the smallest sufficient next step - - Use explicit tie-break rules rather than β€œvibes” - - When uncertain, fail safe by asking for missing info - -inputSchema: - type: object - properties: - userRequest: - type: string - agentRegistry: - type: array - items: - type: object - properties: - name: { type: string } - title: { type: string } - module: { type: string } - capabilities: { type: array, items: { type: string } } - constraints: { type: array, items: { type: string } } - examplePrompts: { type: array, items: { type: string } } - required: [name, title, module] - context: - type: object - required: [userRequest, agentRegistry] - -outputSchema: - type: object - properties: - decision: - type: string - enum: [route, ask_clarifying_question] - agent: - type: string - confidence: - type: number - minimum: 0 - maximum: 1 - reason: - type: string - missingInputs: - type: array - items: { type: string } - suggestedNextPrompt: - type: string - required: [decision, confidence, reason, missingInputs, suggestedNextPrompt] - -examplePrompts: - - "Route this request: 'Has this paper been cited or discredited?'" - - "Route this request: 'Draft my Related Work section from these notes.'" - - "Route this request: 'My biblatex build has undefined citations and an empty bibliography.'" - -owner: PaperKit ---- - -# Orchestrator Instructions - -You are the Orchestrator. Your job is ONLY to choose the best single agent. - -Rules: -- Do NOT solve the task. -- Choose exactly one agent from agentRegistry OR ask_clarifying_question. -- If required inputs are missing, ask_clarifying_question and list missingInputs. -- Output must be valid JSON matching outputSchema, with no extra keys. - -Tie-break rules: -- "peer reviewed / cited by / discredited / provenance / credibility" β†’ librarian -- "help me understand / explain / distil / teach" β†’ tutor -- "derive / implement / debug / algorithm / model" β†’ problem-solver -- "harvard / bibtex / biblatex / biber / citations / doi" β†’ reference-manager -- "latex compile / .tex error / package / build log" β†’ latex-assembler -- "outline / structure / argument flow" β†’ paper-architect -- "draft section / write introduction/methods/related work" β†’ section-drafter -- "polish / rewrite / tighten" β†’ quality-refiner -- otherwise: research-consolidator (if synthesis) or brainstorm (if ideation) \ No newline at end of file +# Orchestrator Agent + +You are **Avery**, the Orchestrator for PaperKit. Your ONLY job is to route the user's request to exactly one specialist agent from the provided registry. You do NOT perform the user's task yourself. + +## Core Behaviour + +1. **Never do the user's task** β€” You only route requests. +2. **Choose exactly one agent** from the provided `agentRegistry`, OR output `ask_clarifying_question` if required inputs are missing. +3. **Output a machine-parseable routing decision** in JSON format. + +## Output Schema + +Your output MUST be valid JSON with exactly these fields: + +```json +{ + "decision": "route" | "ask_clarifying_question", + "agent": "", + "confidence": <0.00 to 1.00>, + "reason": "", + "missingInputs": ["", ...], + "suggestedNextPrompt": "" +} +``` + +- If `decision` is `route`, `agent` must be one of the registry agents. +- If `decision` is `ask_clarifying_question`, `agent` should be empty string. +- `missingInputs` is an array (can be empty if nothing is missing). +- Do NOT add extra keys. + +## Tie-Break Rules + +When multiple agents could handle a request, use these explicit rules in order: + +### Priority 1: Capability Match +Choose the agent whose capabilities most directly match the user's request. + +### Priority 2: Keyword-Based Routing +| Keywords in request | Route to | +|---------------------|----------| +| peer reviewed, cited by, discredited, provenance, credibility, verify source | `librarian` | +| help me understand, explain, teach, clarify concept | `tutor` | +| derive, implement, debug, algorithm, model, calculate | `problem-solver` | +| harvard, bibtex, biblatex, biber, citations, doi, reference format | `reference-manager` | +| latex compile, .tex error, package, build log, pdflatex | `latex-assembler` | +| outline, structure, argument flow, paper organization | `paper-architect` | +| draft section, write introduction, write methods, write related work | `section-drafter` | +| polish, rewrite, tighten, improve prose, academic tone | `quality-refiner` | +| synthesize research, consolidate sources, literature review | `research-consolidator` | +| brainstorm, ideation, generate ideas, explore options | `brainstorm` | + +### Priority 3: Module Match +- Research/writing tasks β†’ prefer `core` agents +- Exploratory/support tasks β†’ prefer `specialist` agents + +### Priority 4: Example Prompt Match +Check if the user's request closely matches an agent's `examplePrompts`. + +## When to Ask for Clarification + +Output `ask_clarifying_question` when: +- The request is ambiguous between multiple agents +- Required context is missing (e.g., which section, what topic) +- The request doesn't match any agent's capabilities + +## Academic Integrity + +- Never fabricate citations, sources, quotes, or attribution +- Flag uncertainties for verification +- Respect academic integrity constraints in all routing decisions diff --git a/.paperkit/tools/check-agents.py b/.paperkit/tools/check-agents.py new file mode 100644 index 0000000..6b30ae1 --- /dev/null +++ b/.paperkit/tools/check-agents.py @@ -0,0 +1,354 @@ +#!/usr/bin/env python3 +""" +Unified Agent System Checker + +Validates the PaperKit agent system for: +1. Schema compliance of all YAML agent definitions +2. Duplicate agent names across files +3. Path reference integrity (YAML path fields point to existing files) +4. Manifest consistency + +Usage: + python3 check-agents.py [--ci] [--verbose] + +Options: + --ci Exit with non-zero code on any error + --verbose Show detailed validation information + +Exit codes: + 0 = All checks passed + 1 = Validation errors found +""" + +import sys +import json +import yaml +import argparse +from pathlib import Path +from typing import Dict, List, Tuple, Optional + +# Color codes for terminal output +class Colors: + BLUE = '\033[0;34m' + GREEN = '\033[0;32m' + RED = '\033[0;31m' + YELLOW = '\033[1;33m' + CYAN = '\033[0;36m' + NC = '\033[0m' + BOLD = '\033[1m' + + +def color(text: str, color_code: str) -> str: + """Wrap text in color codes.""" + return f"{color_code}{text}{Colors.NC}" + + +def find_project_root() -> Optional[Path]: + """Find the project root by looking for .paperkit/ directory.""" + current = Path.cwd() + for path in [current] + list(current.parents): + if (path / ".paperkit").is_dir(): + return path + if (path / "paperkit").is_file(): + return path + return None + + +def load_json_schema(schema_path: Path) -> Optional[Dict]: + """Load JSON Schema from file.""" + try: + with open(schema_path, 'r') as f: + return json.load(f) + except Exception as e: + print(color(f"Error loading schema {schema_path}: {e}", Colors.RED)) + return None + + +def load_yaml_file(yaml_path: Path) -> Optional[Dict]: + """Load YAML file.""" + try: + with open(yaml_path, 'r') as f: + return yaml.safe_load(f) + except Exception as e: + print(color(f" Error loading {yaml_path.name}: {e}", Colors.RED)) + return None + + +def validate_against_schema(data: Dict, schema: Dict, verbose: bool = False) -> List[str]: + """Validate data against JSON Schema. Returns list of error messages.""" + try: + from jsonschema import Draft7Validator + except ImportError: + print(color("Error: jsonschema package not installed", Colors.RED)) + print("Install with: pip install 'jsonschema>=4.0'") + sys.exit(1) + + errors = [] + validator = Draft7Validator(schema) + + for error in sorted(validator.iter_errors(data), key=lambda e: e.path): + path = ".".join(str(p) for p in error.path) if error.path else "(root)" + errors.append(f" [{path}] {error.message}") + + return errors + + +def check_yaml_schema_compliance(project_root: Path, verbose: bool = False) -> Tuple[int, int, List[str]]: + """Validate all YAML agent files against schema.""" + schema_path = project_root / ".paperkit/_cfg/schemas/agent-schema.json" + agents_dir = project_root / ".paperkit/_cfg/agents" + + schema = load_json_schema(schema_path) + if not schema: + return 0, 1, ["Schema file not found"] + + print(color("\n1. YAML Schema Compliance", Colors.BOLD)) + print("-" * 50) + + passed = 0 + failed = 0 + all_errors = [] + + for yaml_file in sorted(agents_dir.glob("*.yaml")): + data = load_yaml_file(yaml_file) + if data is None: + failed += 1 + all_errors.append(f"{yaml_file.name}: Failed to load") + continue + + errors = validate_against_schema(data, schema, verbose) + + if errors: + failed += 1 + print(color(f"βœ— {yaml_file.name}", Colors.RED)) + for err in errors: + print(color(err, Colors.RED)) + all_errors.append(f"{yaml_file.name}: {err}") + else: + passed += 1 + if verbose: + print(color(f"βœ“ {yaml_file.name}", Colors.GREEN)) + + if not verbose and passed > 0: + print(color(f"βœ“ {passed} YAML files valid", Colors.GREEN)) + + return passed, failed, all_errors + + +def check_duplicate_names(project_root: Path, verbose: bool = False) -> Tuple[int, int, List[str]]: + """Check for duplicate agent names.""" + agents_dir = project_root / ".paperkit/_cfg/agents" + + print(color("\n2. Duplicate Agent Name Check", Colors.BOLD)) + print("-" * 50) + + names: Dict[str, List[str]] = {} + errors = [] + + for yaml_file in sorted(agents_dir.glob("*.yaml")): + data = load_yaml_file(yaml_file) + if data and 'name' in data: + name = data['name'] + if name not in names: + names[name] = [] + names[name].append(yaml_file.name) + + duplicates = {k: v for k, v in names.items() if len(v) > 1} + + if duplicates: + for name, files in duplicates.items(): + msg = f"Duplicate name '{name}' in: {', '.join(files)}" + print(color(f"βœ— {msg}", Colors.RED)) + errors.append(msg) + return 0, len(duplicates), errors + else: + print(color(f"βœ“ No duplicate agent names found ({len(names)} unique)", Colors.GREEN)) + return len(names), 0, [] + + +def check_path_references(project_root: Path, verbose: bool = False) -> Tuple[int, int, List[str]]: + """Check that YAML path fields point to existing files.""" + agents_dir = project_root / ".paperkit/_cfg/agents" + + print(color("\n3. Path Reference Integrity", Colors.BOLD)) + print("-" * 50) + + valid = 0 + invalid = 0 + errors = [] + + for yaml_file in sorted(agents_dir.glob("*.yaml")): + data = load_yaml_file(yaml_file) + if data and 'path' in data: + path_value = data['path'] + # Remove leading ./ if present, but preserve the full path + if path_value.startswith('./'): + path_value = path_value[2:] + ref_path = project_root / path_value + + if ref_path.exists(): + valid += 1 + if verbose: + print(color(f"βœ“ {yaml_file.name} -> {data['path']}", Colors.GREEN)) + else: + invalid += 1 + msg = f"{yaml_file.name}: Path not found: {data['path']}" + print(color(f"βœ— {msg}", Colors.RED)) + errors.append(msg) + + if not verbose and valid > 0 and invalid == 0: + print(color(f"βœ“ All {valid} path references valid", Colors.GREEN)) + + return valid, invalid, errors + + +def check_manifest_consistency(project_root: Path, verbose: bool = False) -> Tuple[int, int, List[str]]: + """Check that manifest lists all agents in the agents directory.""" + manifest_path = project_root / ".paperkit/_cfg/agent-manifest.yaml" + agents_dir = project_root / ".paperkit/_cfg/agents" + + print(color("\n4. Manifest Consistency", Colors.BOLD)) + print("-" * 50) + + errors = [] + + manifest = load_yaml_file(manifest_path) + if not manifest or 'agents' not in manifest: + msg = "Manifest not found or missing 'agents' key" + print(color(f"βœ— {msg}", Colors.RED)) + return 0, 1, [msg] + + # Get agent names from manifest + manifest_names = {agent['name'] for agent in manifest['agents']} + + # Get agent names from YAML files + yaml_names = set() + for yaml_file in agents_dir.glob("*.yaml"): + data = load_yaml_file(yaml_file) + if data and 'name' in data: + yaml_names.add(data['name']) + + # Check for agents in YAML but not in manifest + missing_from_manifest = yaml_names - manifest_names + # Check for agents in manifest but not in YAML + missing_from_yaml = manifest_names - yaml_names + + issues = 0 + + if missing_from_manifest: + for name in sorted(missing_from_manifest): + msg = f"Agent '{name}' in YAML but missing from manifest" + print(color(f"βœ— {msg}", Colors.RED)) + errors.append(msg) + issues += len(missing_from_manifest) + + if missing_from_yaml: + for name in sorted(missing_from_yaml): + msg = f"Agent '{name}' in manifest but no YAML file" + print(color(f"βœ— {msg}", Colors.RED)) + errors.append(msg) + issues += len(missing_from_yaml) + + if issues == 0: + print(color(f"βœ“ Manifest matches YAML directory ({len(yaml_names)} agents)", Colors.GREEN)) + return len(yaml_names), 0, [] + + return len(yaml_names) - issues, issues, errors + + +def check_md_file_existence(project_root: Path, verbose: bool = False) -> Tuple[int, int, List[str]]: + """Check that all agents have corresponding MD files in core/specialist.""" + agents_dir = project_root / ".paperkit/_cfg/agents" + core_agents = project_root / ".paperkit/core/agents" + specialist_agents = project_root / ".paperkit/specialist/agents" + + print(color("\n5. MD File Coverage", Colors.BOLD)) + print("-" * 50) + + errors = [] + found = 0 + missing = 0 + + for yaml_file in sorted(agents_dir.glob("*.yaml")): + data = load_yaml_file(yaml_file) + if not data: + continue + + name = data.get('name', yaml_file.stem) + module = data.get('module', 'core') + + if module == 'core': + md_path = core_agents / f"{name}.md" + else: + md_path = specialist_agents / f"{name}.md" + + if md_path.exists(): + found += 1 + if verbose: + print(color(f"βœ“ {name}.md exists", Colors.GREEN)) + else: + missing += 1 + msg = f"{name}.md not found (expected at {md_path.relative_to(project_root)})" + print(color(f"βœ— {msg}", Colors.RED)) + errors.append(msg) + + if not verbose and found > 0 and missing == 0: + print(color(f"βœ“ All {found} MD files exist", Colors.GREEN)) + + return found, missing, errors + + +def main(): + parser = argparse.ArgumentParser(description="Unified Agent System Checker") + parser.add_argument('--ci', action='store_true', help='Exit with error code on failures') + parser.add_argument('--verbose', '-v', action='store_true', help='Show detailed output') + args = parser.parse_args() + + project_root = find_project_root() + if not project_root: + print(color("Error: Could not find PaperKit project root", Colors.RED)) + sys.exit(1) + + print(color("╔═══════════════════════════════════════════════════╗", Colors.CYAN)) + print(color("β•‘ PaperKit Unified Agent System Checker β•‘", Colors.CYAN)) + print(color("β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•", Colors.CYAN)) + print(color(f"Project: {project_root}", Colors.BLUE)) + + total_passed = 0 + total_failed = 0 + all_errors = [] + + # Run all checks + checks = [ + check_yaml_schema_compliance, + check_duplicate_names, + check_path_references, + check_manifest_consistency, + check_md_file_existence, + ] + + for check_func in checks: + passed, failed, errors = check_func(project_root, args.verbose) + total_passed += passed + total_failed += failed + all_errors.extend(errors) + + # Summary + print(color("\n" + "=" * 50, Colors.BOLD)) + print(color("SUMMARY", Colors.BOLD)) + print("=" * 50) + print(f" Total Passed: {color(str(total_passed), Colors.GREEN)}") + print(f" Total Failed: {color(str(total_failed), Colors.RED if total_failed > 0 else Colors.GREEN)}") + + if total_failed > 0: + print(color(f"\n⚠ {total_failed} issue(s) found", Colors.YELLOW)) + if args.ci: + sys.exit(1) + else: + print(color("\nβœ“ All checks passed!", Colors.GREEN)) + + sys.exit(0) + + +if __name__ == "__main__": + main() diff --git a/.paperkit/tools/validate-agent-schema.py b/.paperkit/tools/validate-agent-schema.py index 988efca..49d918a 100755 --- a/.paperkit/tools/validate-agent-schema.py +++ b/.paperkit/tools/validate-agent-schema.py @@ -13,8 +13,8 @@ --agent AGENT_NAME Validate specific agent only --verbose Show detailed validation information --ci CI mode: exit with error code on validation failure - --schema PATH Path to JSON Schema file (overrides default) - --agents-dir PATH Path to agents directory (overrides default) + --schema PATH Path to JSON Schema file (default: .paperkit/_cfg/schemas/agent-schema.json) + --agents-dir PATH Path to agents directory (default: .paperkit/_cfg/agents) Environment Variables: PAPERKIT_SCHEMA_PATH Override default schema path @@ -114,8 +114,8 @@ def main(): parser.add_argument('--agent', help='Validate specific agent only') parser.add_argument('--verbose', action='store_true', help='Show detailed information') parser.add_argument('--ci', action='store_true', help='CI mode: exit with error on failure') - parser.add_argument('--schema', help='Path to JSON Schema file (default: .paper/_cfg/schemas/agent-schema.json)') - parser.add_argument('--agents-dir', help='Path to agents directory (default: .paper/_cfg/agents)') + parser.add_argument('--schema', help='Path to JSON Schema file (default: .paperkit/_cfg/schemas/agent-schema.json)') + parser.add_argument('--agents-dir', help='Path to agents directory (default: .paperkit/_cfg/agents)') args = parser.parse_args() # Paths - support environment variables and command-line overrides @@ -125,13 +125,13 @@ def main(): schema_path = Path(args.schema) else: schema_path = Path(os.getenv('PAPERKIT_SCHEMA_PATH', - repo_root / ".paper" / "_cfg" / "schemas" / "agent-schema.json")) + repo_root / ".paperkit" / "_cfg" / "schemas" / "agent-schema.json")) if args.agents_dir: agents_dir = Path(args.agents_dir) else: agents_dir = Path(os.getenv('PAPERKIT_AGENTS_DIR', - repo_root / ".paper" / "_cfg" / "agents")) + repo_root / ".paperkit" / "_cfg" / "agents")) print("Agent Schema Validation") print("=" * 70) diff --git a/.paperkit/tools/validate.py b/.paperkit/tools/validate.py index b9c71ec..c7fc47a 100644 --- a/.paperkit/tools/validate.py +++ b/.paperkit/tools/validate.py @@ -61,12 +61,12 @@ def check_dependencies() -> bool: def find_project_root() -> Optional[Path]: - """Find the project root by looking for .paper/ directory.""" + """Find the project root by looking for .paperkit/ directory.""" current = Path.cwd() # Check current directory and parents for path in [current] + list(current.parents): - if (path / ".paper").is_dir(): + if (path / ".paperkit").is_dir(): return path if (path / "paperkit").is_file(): return path @@ -157,7 +157,7 @@ def validate_agents(project_root: Path, verbose: bool = False) -> Tuple[int, int """Validate all agent definitions. Returns (passed, failed) counts.""" import yaml - schema_path = project_root / ".paper/_cfg/schemas/agent-schema.json" + schema_path = project_root / ".paperkit/_cfg/schemas/agent-schema.json" schema = load_json_schema(schema_path) if not schema: @@ -172,8 +172,8 @@ def validate_agents(project_root: Path, verbose: bool = False) -> Tuple[int, int # Check both core and specialist agents agent_dirs = [ - project_root / ".paper/core/agents", - project_root / ".paper/specialist/agents" + project_root / ".paperkit/core/agents", + project_root / ".paperkit/specialist/agents" ] for agent_dir in agent_dirs: @@ -205,7 +205,7 @@ def validate_agents(project_root: Path, verbose: bool = False) -> Tuple[int, int def validate_workflows(project_root: Path, verbose: bool = False) -> Tuple[int, int]: """Validate all workflow definitions. Returns (passed, failed) counts.""" - schema_path = project_root / ".paper/_cfg/schemas/workflow-schema.json" + schema_path = project_root / ".paperkit/_cfg/schemas/workflow-schema.json" schema = load_json_schema(schema_path) if not schema: @@ -220,9 +220,9 @@ def validate_workflows(project_root: Path, verbose: bool = False) -> Tuple[int, # Check both core and specialist workflows workflow_dirs = [ - project_root / ".paper/core/workflows", - project_root / ".paper/specialist/workflows", - project_root / ".paper/_cfg/workflows" + project_root / ".paperkit/core/workflows", + project_root / ".paperkit/specialist/workflows", + project_root / ".paperkit/_cfg/workflows" ] for workflow_dir in workflow_dirs: @@ -255,7 +255,7 @@ def validate_workflows(project_root: Path, verbose: bool = False) -> Tuple[int, def validate_tools(project_root: Path, verbose: bool = False) -> Tuple[int, int]: """Validate all tool definitions. Returns (passed, failed) counts.""" - schema_path = project_root / ".paper/_cfg/schemas/tool-schema.json" + schema_path = project_root / ".paperkit/_cfg/schemas/tool-schema.json" schema = load_json_schema(schema_path) if not schema: @@ -268,7 +268,7 @@ def validate_tools(project_root: Path, verbose: bool = False) -> Tuple[int, int] passed = 0 failed = 0 - tool_dir = project_root / ".paper/_cfg/tools" + tool_dir = project_root / ".paperkit/_cfg/tools" if not tool_dir.is_dir(): print(color(" No tool definitions directory found", Colors.YELLOW)) @@ -313,7 +313,7 @@ def validate_manifests(project_root: Path, verbose: bool = False) -> Tuple[int, ] for manifest_name, required_keys in manifests: - manifest_path = project_root / f".paper/_cfg/{manifest_name}" + manifest_path = project_root / f".paperkit/_cfg/{manifest_name}" if not manifest_path.exists(): print(color(f"⚠ {manifest_name}: Not found", Colors.YELLOW)) @@ -341,7 +341,7 @@ def validate_manifests(project_root: Path, verbose: bool = False) -> Tuple[int, def validate_ide_sync(project_root: Path, verbose: bool = False) -> Tuple[int, int]: - """Check if IDE files are in sync with .paper/ source. Returns (passed, failed) counts.""" + """Check if IDE files are in sync with .paperkit/ source. Returns (passed, failed) counts.""" print(color("\nπŸ“‹ Checking IDE File Sync", Colors.BOLD)) print("-" * 40) @@ -350,7 +350,7 @@ def validate_ide_sync(project_root: Path, verbose: bool = False) -> Tuple[int, i # Find all agent definitions agent_names = [] - for agent_dir in [project_root / ".paper/core/agents", project_root / ".paper/specialist/agents"]: + for agent_dir in [project_root / ".paperkit/core/agents", project_root / ".paperkit/specialist/agents"]: if agent_dir.is_dir(): for f in agent_dir.glob("*.md"): # Extract name from filename (paper-xxx.md -> xxx) diff --git a/docs/dev/agent-audit/baseline.txt b/docs/dev/agent-audit/baseline.txt new file mode 100644 index 0000000..f938da0 --- /dev/null +++ b/docs/dev/agent-audit/baseline.txt @@ -0,0 +1,29 @@ +Agent Schema Validation +====================================================================== + +Validating 11 agent(s)... + +βœ“ brainstorm VALID +βœ“ latex-assembler VALID +βœ“ librarian VALID +βœ— orchestrator INVALID + [root] 'identity' is a required property + [root] 'path' is a required property + [root] Additional properties are not allowed ('decisionSchema', 'instructions', 'schemaVersion', 'status' were unexpected) + +βœ“ paper-architect VALID +βœ“ problem-solver VALID +βœ“ quality-refiner VALID +βœ“ reference-manager VALID +βœ“ research-consolidator VALID +βœ“ section-drafter VALID +βœ“ tutor VALID +====================================================================== + +Validation Summary: + Total agents: 11 + Valid: 10 + Invalid: 1 + Total errors: 3 + +⚠️ Validation failed with 1 invalid agent(s) diff --git a/docs/dev/agent-audit/decision.md b/docs/dev/agent-audit/decision.md new file mode 100644 index 0000000..69ec6db --- /dev/null +++ b/docs/dev/agent-audit/decision.md @@ -0,0 +1,52 @@ +# Agent System Unification Decision + +Generated: 2026-01-06 +Updated: 2026-01-06 + +## Decision + +**Canonical Design** β€” Two-file split: + +1. **`.paperkit/_cfg/agents/*.yaml`** = Schema-validated **metadata ONLY** + - Contains all fields defined in `agent-schema.json` + - Machine-readable, validated by CI + - `path` field points to the corresponding MD file + +2. **`.paperkit/{core,specialist}/agents/*.md`** = **Prompt/instructions content** + - Contains the agent's behavioural instructions + - Referenced by `path` field in the YAML file + - Used by runtime/loaders to activate the agent persona + +## Rationale + +### Schema Design + +The `agent-schema.json` defines what metadata belongs in YAML files: +- Required: `name`, `displayName`, `title`, `icon`, `identity`, `module`, `path` +- Optional: `version`, `capabilities`, `constraints`, `principles`, `inputSchema`, `outputSchema`, `examplePrompts`, `owner` +- `additionalProperties: false` β€” No extra keys allowed + +The `path` property has a regex pattern that expects `.paperkit/(core|specialist)/agents/*.md` files. + +### Why This Split? + +1. **Separation of concerns**: Metadata (structured, validatable) vs. Instructions (prose, prompts) +2. **CI validation**: YAML files can be schema-validated without parsing markdown +3. **Runtime loading**: MD files contain the actual agent behavior loaded at runtime +4. **No duplication**: Metadata lives in YAML, instructions live in MD + +### What Changed + +1. **orchestrator.yaml**: Now contains ONLY schema-compliant metadata (no `instructions`, `decisionSchema`, etc.) +2. **orchestrator.md**: Now contains ONLY the agent's behavioral instructions (no YAML frontmatter) +3. **Validation**: Both `validate-agent-schema.py` and `check-agents.py` verify the system integrity + +## Implementation Status + +- [x] `orchestrator.yaml` is schema-compliant +- [x] `orchestrator.md` contains behavioral instructions +- [x] All 11 agent YAML files pass validation +- [x] All 11 agent MD files exist and are referenced correctly +- [x] `validate.py` uses correct `.paperkit/` paths +- [x] `check-agents.py` provides unified validation +- [x] CI workflow runs both validation checks diff --git a/docs/dev/agent-audit/how-agents-are-structured.md b/docs/dev/agent-audit/how-agents-are-structured.md new file mode 100644 index 0000000..76f5b0d --- /dev/null +++ b/docs/dev/agent-audit/how-agents-are-structured.md @@ -0,0 +1,173 @@ +# How Agents Are Structured + +This document explains the PaperKit agent system structure and how the dual-file approach works. + +## Overview + +PaperKit uses a dual-file approach for agent definitions: + +1. **YAML Metadata Files** (`.paperkit/_cfg/agents/*.yaml`) - Schema-validated metadata **ONLY** +2. **Markdown Agent Files** (`.paperkit/{core,specialist}/agents/*.md`) - Behavioural instructions and prompts + +## Directory Structure + +``` +.paperkit/ +β”œβ”€β”€ _cfg/ +β”‚ β”œβ”€β”€ agents/ # YAML metadata files (schema-validated) +β”‚ β”‚ β”œβ”€β”€ orchestrator.yaml +β”‚ β”‚ β”œβ”€β”€ paper-architect.yaml +β”‚ β”‚ └── ... +β”‚ β”œβ”€β”€ schemas/ +β”‚ β”‚ └── agent-schema.json # JSON Schema for validation +β”‚ └── agent-manifest.yaml # Index of all agents +β”œβ”€β”€ core/ +β”‚ └── agents/ # Core agent instructions (MD) +β”‚ β”œβ”€β”€ orchestrator.md +β”‚ β”œβ”€β”€ paper-architect.md +β”‚ └── ... +└── specialist/ + └── agents/ # Specialist agent instructions (MD) + β”œβ”€β”€ brainstorm.md + β”œβ”€β”€ tutor.md + └── ... +``` + +## YAML Metadata Files + +Location: `.paperkit/_cfg/agents/` + +These files contain **metadata only** β€” no instructions or prompts. They are validated against `agent-schema.json`. + +### Required Fields + +- `name` - Machine identifier (kebab-case) +- `displayName` - Human persona name +- `title` - Functional title +- `icon` - Emoji representation +- `module` - Either "core" or "specialist" +- `identity` - Object with `role` and `description` +- `path` - Reference to the MD file (must match pattern) + +### Optional Fields + +- `version` - Semantic version (e.g., "1.0.0") +- `capabilities` - Array of strings +- `constraints` - Array of strings +- `principles` - Array of strings +- `inputSchema` / `outputSchema` - JSON Schema objects +- `examplePrompts` - Array of example prompt strings +- `owner` - Maintainer name + +### Example YAML File + +```yaml +name: paper-architect +displayName: Morgan +title: Paper Architect +icon: πŸ—οΈ +module: core + +identity: + role: System Architect + description: > + Transforms paper scope and goals into comprehensive, hierarchical paper + structure with logical skeletons and LaTeX scaffolding. + communicationStyle: Calm, pragmatic tones. + +capabilities: + - Design comprehensive paper structures + - Create hierarchical section organization + +constraints: + - Cannot start writing content (delegates to Section Drafter) + +path: .paperkit/core/agents/paper-architect.md +``` + +## Markdown Agent Files + +Location: `.paperkit/{core,specialist}/agents/` + +These files contain the **behavioural instructions** for the agent: + +- Agent persona and role description +- How the agent should respond +- Specific rules and tie-break logic +- Output format requirements +- Menu systems and workflows (if applicable) + +**Important**: MD files should NOT duplicate the YAML metadata. They contain only the instructions/prompts that define the agent's behavior. + +The runtime (`generate.sh`) reads from these files to generate IDE-specific agent files. + +## Why This Split? + +| Concern | YAML Files | MD Files | +|---------|------------|----------| +| Schema validation | βœ… Validated by CI | Not validated | +| Structured metadata | βœ… Easy to parse | N/A | +| Agent instructions | ❌ None | βœ… Full prompts | +| IDE generation | Metadata reference | βœ… Used by generate.sh | +| Runtime loading | Metadata lookup | βœ… Behavior source | + +## Validation + +### Schema Compliance + +All YAML files must comply with `.paperkit/_cfg/schemas/agent-schema.json`: + +```bash +python .paperkit/tools/validate-agent-schema.py --ci +``` + +### Unified System Check + +Run the comprehensive check to validate: +- Schema compliance +- No duplicate agent names +- Path references exist +- Manifest consistency +- MD file coverage + +```bash +python .paperkit/tools/check-agents.py --ci +``` + +## Adding a New Agent + +1. **Determine module**: `core` (paper writing) or `specialist` (support) + +2. **Create the MD file** with instructions: + ``` + .paperkit/{core|specialist}/agents/{agent-name}.md + ``` + +3. **Create the YAML metadata file** (schema-compliant): + ``` + .paperkit/_cfg/agents/{agent-name}.yaml + ``` + +4. **Add to manifest**: + Edit `.paperkit/_cfg/agent-manifest.yaml` to include the new agent + +5. **Validate**: + ```bash + python .paperkit/tools/check-agents.py --ci + ``` + +6. **Generate IDE files** (optional): + ```bash + .paperkit/tools/generate.sh + ``` + +## Academic Integrity + +All agents follow these critical principles: + +- Academic integrity is paramount +- Proper attribution and Harvard-style citations required +- Never fabricate citations or sources +- Flag uncertainties for verification + +These constraints are embedded in agent definitions and cannot be bypassed. diff --git a/docs/dev/agent-audit/inventory.md b/docs/dev/agent-audit/inventory.md new file mode 100644 index 0000000..72b3211 --- /dev/null +++ b/docs/dev/agent-audit/inventory.md @@ -0,0 +1,72 @@ +# Agent System Inventory + +Generated: 2026-01-06 + +## Overview + +This document inventories all agent definition files in PaperKit. + +## Dual System Discovery + +PaperKit has **two parallel agent definition systems**: + +### System 1: YAML Metadata Files +Location: `.paperkit/_cfg/agents/` + +| File | Schema-Compliant | Notes | +|------|------------------|-------| +| brainstorm.yaml | βœ… | Points to specialist/agents/brainstorm.md | +| latex-assembler.yaml | βœ… | Points to core/agents/latex-assembler.md | +| librarian.yaml | βœ… | Points to specialist/agents/librarian.md | +| orchestrator.yaml | ❌ | Missing: identity, path; Extra: decisionSchema, instructions, schemaVersion, status | +| paper-architect.yaml | βœ… | Points to core/agents/paper-architect.md | +| problem-solver.yaml | βœ… | Points to core/agents/problem-solver.md | +| quality-refiner.yaml | βœ… | Points to core/agents/quality-refiner.md | +| reference-manager.yaml | βœ… | Points to core/agents/reference-manager.md | +| research-consolidator.yaml | βœ… | Points to core/agents/research-consolidator.md | +| section-drafter.yaml | βœ… | Points to core/agents/section-drafter.md | +| tutor.yaml | βœ… | Points to specialist/agents/tutor.md | + +### System 2: Markdown Agent Files (with YAML Frontmatter) +Location: `.paperkit/{core,specialist}/agents/` + +#### Core Agents +| File | Has Frontmatter | Notes | +|------|-----------------|-------| +| latex-assembler.md | βœ… | Operational instructions | +| orchestrator.md | βœ… | Full schema-compliant metadata + instructions | +| paper-architect.md | βœ… | Operational instructions | +| quality-refiner.md | βœ… | Operational instructions | +| reference-manager.md | βœ… | Operational instructions | +| research-consolidator.md | βœ… | Operational instructions | +| section-drafter.md | βœ… | Operational instructions | + +#### Specialist Agents +| File | Has Frontmatter | Notes | +|------|-----------------|-------| +| brainstorm.md | βœ… | Operational instructions | +| librarian.md | βœ… | Operational instructions | +| problem-solver.md | βœ… | Operational instructions | +| tutor.md | βœ… | Operational instructions | + +## Manifest Files + +| File | Location | Purpose | +|------|----------|---------| +| agent-manifest.yaml | `.paperkit/_cfg/` | Index of all agents with paths to YAML files | +| manifest.yaml | `.paperkit/_cfg/` | Master manifest pointing to schemas | + +## Schema Files + +| File | Location | Used By | +|------|----------|---------| +| agent-schema.json | `.paperkit/_cfg/schemas/` | validate-agent-schema.py | + +## Key Findings + +1. **Duplication exists**: Each agent has BOTH a `.yaml` metadata file AND a `.md` definition file +2. **Schema expects MD files**: The `path` property regex requires `.paperkit/(core|specialist)/agents/*.md` +3. **Runtime uses MD files**: `generate.sh` scans `.paperkit/{core,specialist}/agents/*.md` +4. **Validation checks YAML**: `validate-agent-schema.py` checks `.paperkit/_cfg/agents/*.yaml` +5. **YAML files reference MD files**: Each YAML file has a `path` field pointing to the MD file +6. **orchestrator.yaml is non-compliant**: Missing required fields and has extra properties diff --git a/docs/dev/agent-audit/paperkit-tree.txt b/docs/dev/agent-audit/paperkit-tree.txt new file mode 100644 index 0000000..f8bd32f --- /dev/null +++ b/docs/dev/agent-audit/paperkit-tree.txt @@ -0,0 +1,142 @@ +.paperkit +β”œβ”€β”€ _cfg +β”‚Β Β  β”œβ”€β”€ agent-manifest.yaml +β”‚Β Β  β”œβ”€β”€ agents +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ README.md +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ brainstorm.yaml +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ latex-assembler.yaml +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ librarian.yaml +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ orchestrator.yaml +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ paper-architect.yaml +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ problem-solver.yaml +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ quality-refiner.yaml +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ reference-manager.yaml +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ research-consolidator.yaml +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ section-drafter.yaml +β”‚Β Β  β”‚Β Β  └── tutor.yaml +β”‚Β Β  β”œβ”€β”€ guides +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ README.md +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ acadmic-integrity-guide.md +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ harvard-citation-guide.md +β”‚Β Β  β”‚Β Β  └── locking-system.md +β”‚Β Β  β”œβ”€β”€ ides +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ codex.yaml +β”‚Β Β  β”‚Β Β  └── github-copilot.yaml +β”‚Β Β  β”œβ”€β”€ manifest.yaml +β”‚Β Β  β”œβ”€β”€ resources +β”‚Β Β  β”‚Β Β  └── citation-rules.yaml +β”‚Β Β  β”œβ”€β”€ routing.registry.yaml +β”‚Β Β  β”œβ”€β”€ schemas +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ README.md +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ agent-schema.json +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ tool-schema.json +β”‚Β Β  β”‚Β Β  └── workflow-schema.json +β”‚Β Β  β”œβ”€β”€ tool-manifest.yaml +β”‚Β Β  β”œβ”€β”€ tools +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ README.md +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ build-latex.yaml +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ check-dependencies.yaml +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ extract-evidence.yaml +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ format-references.yaml +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ lint-latex.yaml +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ lock-chapter.yaml +β”‚Β Β  β”‚Β Β  └── validate-structure.yaml +β”‚Β Β  β”œβ”€β”€ validation +β”‚Β Β  β”‚Β Β  └── README.md +β”‚Β Β  β”œβ”€β”€ version.yaml +β”‚Β Β  β”œβ”€β”€ workflow-manifest.yaml +β”‚Β Β  └── workflows +β”‚Β Β  β”œβ”€β”€ README.md +β”‚Β Β  β”œβ”€β”€ analyze-problem.yaml +β”‚Β Β  β”œβ”€β”€ build.yaml +β”‚Β Β  β”œβ”€β”€ citation-completeness.yaml +β”‚Β Β  β”œβ”€β”€ consolidate.yaml +β”‚Β Β  β”œβ”€β”€ extract-citations.yaml +β”‚Β Β  β”œβ”€β”€ format-bibliography.yaml +β”‚Β Β  β”œβ”€β”€ generate-ideas.yaml +β”‚Β Β  β”œβ”€β”€ lock-chapter.yaml +β”‚Β Β  β”œβ”€β”€ outline.yaml +β”‚Β Β  β”œβ”€β”€ refine-section.yaml +β”‚Β Β  β”œβ”€β”€ review-draft.yaml +β”‚Β Β  β”œβ”€β”€ search-sources.yaml +β”‚Β Β  β”œβ”€β”€ skeleton.yaml +β”‚Β Β  β”œβ”€β”€ validate-citations.yaml +β”‚Β Β  └── write-section.yaml +β”œβ”€β”€ core +β”‚Β Β  β”œβ”€β”€ agents +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ latex-assembler.md +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ orchestrator.md +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ paper-architect.md +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ quality-refiner.md +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ reference-manager.md +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ research-consolidator.md +β”‚Β Β  β”‚Β Β  └── section-drafter.md +β”‚Β Β  └── config.yaml +β”œβ”€β”€ data +β”‚Β Β  β”œβ”€β”€ output-drafts +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ outlines +β”‚Β Β  β”‚Β Β  β”‚Β Β  └── paper_outline.md +β”‚Β Β  β”‚Β Β  └── sections +β”‚Β Β  β”‚Β Β  └── 02_perceptual_foundations_v2.tex +β”‚Β Β  β”œβ”€β”€ output-final +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ build_report.md +β”‚Β Β  β”‚Β Β  └── pdf +β”‚Β Β  β”‚Β Β  └── main.pdf +β”‚Β Β  └── output-refined +β”‚Β Β  β”œβ”€β”€ references +β”‚Β Β  β”‚Β Β  └── reference-status.md +β”‚Β Β  └── research +β”‚Β Β  β”œβ”€β”€ CITATION-VALUE-ANALYSIS.md +β”‚Β Β  β”œβ”€β”€ FINAL-CITATION-MAPPING.md +β”‚Β Β  β”œβ”€β”€ HANDOFF-TO-ARCHITECT.md +β”‚Β Β  β”œβ”€β”€ RESEARCH-CONSOLIDATION-REPORT.md +β”‚Β Β  β”œβ”€β”€ prior-art-synthesis.md +β”‚Β Β  └── technical-documentation-consolidated.md +β”œβ”€β”€ docs +β”‚Β Β  β”œβ”€β”€ COPILOT-COMPARISON.md +β”‚Β Β  β”œβ”€β”€ COPILOT-DIR-GENERATION.md +β”‚Β Β  β”œβ”€β”€ DEPRECATION-NOTICE.md +β”‚Β Β  β”œβ”€β”€ IMPLEMENTATION-COMPLETE.md +β”‚Β Β  β”œβ”€β”€ PR-SUMMARY.md +β”‚Β Β  β”œβ”€β”€ TEST-PLAN-REGENERATION.md +β”‚Β Β  β”œβ”€β”€ TEST-RESULTS-REGENERATION.md +β”‚Β Β  β”œβ”€β”€ codex-instructions.md +β”‚Β Β  β”œβ”€β”€ github-copilot-instructions.md +β”‚Β Β  β”œβ”€β”€ legacy-agent-examples.md +β”‚Β Β  β”œβ”€β”€ release-quick-start.md +β”‚Β Β  └── release-workflow.md +β”œβ”€β”€ open-agent-system +β”‚Β Β  β”œβ”€β”€ LICENSE +β”‚Β Β  β”œβ”€β”€ OpenAgentDefinition.md +β”‚Β Β  └── README.md +β”œβ”€β”€ specialist +β”‚Β Β  β”œβ”€β”€ agents +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ brainstorm.md +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ librarian.md +β”‚Β Β  β”‚Β Β  β”œβ”€β”€ problem-solver.md +β”‚Β Β  β”‚Β Β  └── tutor.md +β”‚Β Β  └── config.yaml +└── tools + β”œβ”€β”€ README.md + β”œβ”€β”€ build-latex.sh + β”œβ”€β”€ bundle.sh + β”œβ”€β”€ check-dependencies.sh + β”œβ”€β”€ extract-evidence.sh + β”œβ”€β”€ format-references.py + β”œβ”€β”€ generate-copilot.sh + β”œβ”€β”€ generate-docs.sh + β”œβ”€β”€ generate.ps1 + β”œβ”€β”€ generate.sh + β”œβ”€β”€ get-title.sh + β”œβ”€β”€ get-version.sh + β”œβ”€β”€ lint-latex.sh + β”œβ”€β”€ lock-chapter.sh + β”œβ”€β”€ test-version-system.sh + β”œβ”€β”€ validate-agent-schema.py + β”œβ”€β”€ validate-structure.py + β”œβ”€β”€ validate-tool-schema.py + β”œβ”€β”€ validate-workflow-schema.py + β”œβ”€β”€ validate.py + └── version-manager.py + +26 directories, 114 files diff --git a/docs/dev/agent-audit/runtime-paths.md b/docs/dev/agent-audit/runtime-paths.md new file mode 100644 index 0000000..614d446 --- /dev/null +++ b/docs/dev/agent-audit/runtime-paths.md @@ -0,0 +1,112 @@ +# Runtime Paths Analysis + +Generated: 2026-01-06 + +## Runtime Agent Loading + +### Primary Loader: `generate.sh` + +**Location**: `.paperkit/tools/generate.sh` + +**Code Path** (lines 196-235): +```bash +# Core agents +for agent_file in "${PAPERKIT_ROOT}"/.paperkit/core/agents/*.md; do + [ -f "$agent_file" ] || continue + if ! generate_copilot_agent "$agent_file"; then + has_errors=true + fi +done + +# Specialist agents +for agent_file in "${PAPERKIT_ROOT}"/.paperkit/specialist/agents/*.md; do + [ -f "$agent_file" ] || continue + if ! generate_copilot_agent "$agent_file"; then + has_errors=true + fi +done +``` + +**Directories Scanned**: +- `.paperkit/core/agents/*.md` +- `.paperkit/specialist/agents/*.md` + +**Expected File Format**: Markdown with YAML frontmatter + +### Validation Script: `validate-agent-schema.py` + +**Location**: `.paperkit/tools/validate-agent-schema.py` + +**Default Paths** (lines 117-118, 127-134): +```python +parser.add_argument('--schema', help='Path to JSON Schema file (default: .paper/_cfg/schemas/agent-schema.json)') +parser.add_argument('--agents-dir', help='Path to agents directory (default: .paper/_cfg/agents)') +``` + +**Note**: The hardcoded defaults use `.paper/` NOT `.paperkit/`. This is overridden via CLI in the workflow. + +**Directories Scanned**: +- `.paperkit/_cfg/agents/*.yaml` (when called with `--agents-dir`) + +### Second Validator: `validate.py` + +**Location**: `.paperkit/tools/validate.py` + +**Hardcoded Paths** (lines 160, 175-176, 353): +```python +schema_path = project_root / ".paper/_cfg/schemas/agent-schema.json" +# ... +agent_dirs = [ + project_root / ".paper/core/agents", + project_root / ".paper/specialist/agents" +] +``` + +**Issue**: This script uses `.paper/` paths which don't exist. It should use `.paperkit/`. + +### CI Workflow: `validate-agent-metadata.yml` + +**Location**: `.github/workflows/validate-agent-metadata.yml` + +**Monitored Paths** (lines 7-10, 14-17): +```yaml +paths: + - '.paperkit/_cfg/agents/**' + - '.paperkit/_cfg/schemas/agent-schema.json' + - '.paperkit/_cfg/agent-manifest.yaml' + - '.paperkit/tools/validate-agent-schema.py' +``` + +**Execution** (lines 44-46): +```yaml +python .paperkit/tools/validate-agent-schema.py --ci \ + --schema .paperkit/_cfg/schemas/agent-schema.json \ + --agents-dir .paperkit/_cfg/agents +``` + +## Schema Path Pattern + +From `agent-schema.json` (line 141): +```json +"path": { + "pattern": "^\\.paperkit/(core|specialist)/agents/[a-z][a-z0-9-]*\\.md$" +} +``` + +**This expects**: `.paperkit/core/agents/*.md` or `.paperkit/specialist/agents/*.md` + +## Summary + +| Component | Reads From | Format | +|-----------|------------|--------| +| generate.sh (runtime) | `.paperkit/{core,specialist}/agents/` | `.md` | +| validate-agent-schema.py (CI) | `.paperkit/_cfg/agents/` | `.yaml` | +| validate.py | `.paper/{core,specialist}/agents/` (BROKEN) | `.md` frontmatter | +| agent-schema.json path regex | `.paperkit/{core,specialist}/agents/` | `.md` | + +## Identified Issues + +1. **validate.py uses wrong base path**: Uses `.paper/` instead of `.paperkit/` +2. **Two validation systems**: One for YAML, one for MD frontmatter +3. **Schema path regex mismatch**: Schema expects MD paths, but YAML files are validated +4. **YAML files act as metadata index**: They point to MD files but aren't the canonical source