diff --git a/.cursor/skills/initiative-sync.md b/.cursor/skills/initiative-sync.md new file mode 100644 index 0000000..9371406 --- /dev/null +++ b/.cursor/skills/initiative-sync.md @@ -0,0 +1,38 @@ + +# /initiative-sync — Board ↔ initiatives.json + +Reconcile GitHub Project boards with `.cursor/state/initiatives.json` and tap-agent portfolio briefs. + +## Usage + +```bash +./bin/initiative-sync.sh status # all projects with boards +./bin/initiative-sync.sh status --project qulib +./bin/initiative-sync.sh pull qulib --apply # board → initiatives.json +./bin/initiative-sync.sh plan notquality # briefs missing from board +``` + +## What it checks + +| Source | Role | +|--------|------| +| **GitHub Project board** | Card column (`Proposal`, `Todo`, `Plans`, …) and issue/PR link | +| **initiatives.json** | Agent session state (phases, metrics, checkpoints) | +| **portfolio/initiatives/*.md** | Planned briefs in tap-agent (flags when board has no card) | + +## When to run + +- Session start (Conductor checklist) — after `gh project item-list`, run `status` +- After `/approve` or manual board moves — `pull --apply` +- When portfolio brief exists but no card — `plan ` then create issue + board item + +## Rules + +- **Pull** updates `board_status`, `github`, and `status` from the board; it does **not** delete local-only initiatives +- **Never** modifies another project's files when run from the wrong cwd — paths come from `scripts/project_registry.py` or `~/.tapagent/portfolio.json` +- Board titles must start with an initiative id (`QLIB-001`, `NQ-001`, …) to auto-link + +## Related + +- ORCH-013 proposal engine (creates `proposal` issues on board) +- [product-conductor.md](../rules/product-conductor.md) — board column lifecycle diff --git a/.cursor/skills/portfolio.md b/.cursor/skills/portfolio.md index 8139313..ed2a50e 100644 --- a/.cursor/skills/portfolio.md +++ b/.cursor/skills/portfolio.md @@ -39,6 +39,8 @@ PORTFOLIO (5 projects, 3 active) Reads `~/.tapagent/portfolio.json` and each project's `initiatives.json`. +Board drift: run `./bin/initiative-sync.sh status` (see [initiative-sync.md](./initiative-sync.md)). + ## Rules - Cross-project reads only — never modify another project's state files diff --git a/CHANGELOG.md b/CHANGELOG.md index 7d3e4e2..f69000e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -23,6 +23,7 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). The - **Autonomous proposal engine (ORCH-013).** GitHub Actions workflow (weekly + on-demand) calls Claude with prompt caching to generate 3–5 initiative proposals per project (qulib, notquality). Posts digest to `#proposals` Slack, creates issues tagged `proposal` on each repo's project board with Proposal status. Human-in-the-loop review via `/refine ` and `/approve` issue comments. Includes `proposal_engine.py`, `proposal_refine.py`, `proposal_approve.py` and matching workflow files. - **Agent rename pass (ORCH-012).** Composer → Builder, Supervisor → Guardian (Cursor agent), Reporter → Inspector; `docs/agent-naming.md` disambiguates Builder vs Orchestrator and Guardian agent vs gates; Planner kept for context isolation. - **Conductor single entry point (ORCH-011).** Documented and ruled that Conductor is the only user-facing Cursor agent; explicit dispatch to Composer, Supervisor, or Reporter via Communicator META handoffs. +- **Initiative board sync.** `bin/initiative-sync.sh` compares GitHub Project boards with per-project `initiatives.json` and tap-agent portfolio briefs; `pull --apply` imports board cards (status, issue links). `/initiative-sync` skill and `scripts/project_registry.py` shared project paths. --- diff --git a/bin/initiative-sync.sh b/bin/initiative-sync.sh new file mode 100755 index 0000000..ed3b5ed --- /dev/null +++ b/bin/initiative-sync.sh @@ -0,0 +1,17 @@ +#!/usr/bin/env bash +# Sync GitHub Project boards with per-project .cursor/state/initiatives.json +# +# Usage: +# ./bin/initiative-sync.sh status [--project qulib|notquality|all] +# ./bin/initiative-sync.sh pull [--apply] +# ./bin/initiative-sync.sh plan +# +# Requires: gh CLI (authenticated). Optional: ~/.tapagent/portfolio.json for paths. + +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" +ROOT="$(dirname "$SCRIPT_DIR")" + +cd "$ROOT/scripts" +exec python3 initiative_sync.py "$@" diff --git a/scripts/initiative_sync.py b/scripts/initiative_sync.py new file mode 100644 index 0000000..0875e1d --- /dev/null +++ b/scripts/initiative_sync.py @@ -0,0 +1,415 @@ +#!/usr/bin/env python3 +""" +Sync GitHub Project boards with per-project .cursor/state/initiatives.json. + +Board is the visual source of truth for *where* a card sits; initiatives.json is +the agent session source of truth for *phases, metrics, and checkpoints*. +This tool reports drift and can pull board state into initiatives.json. + +Usage: + python scripts/initiative_sync.py status [--project qulib|notquality|tap-agent|all] + python scripts/initiative_sync.py pull [--apply] + python scripts/initiative_sync.py plan # portfolio briefs missing from board + +Requires: gh CLI authenticated (GH_TOKEN or gh auth login). +""" + +from __future__ import annotations + +import argparse +import json +import re +import subprocess +import sys +from datetime import datetime, timezone +from pathlib import Path +from typing import Any + +from project_registry import ( + GITHUB_LOGIN, + PORTFOLIO_INITIATIVES_DIR, + PROJECTS, + get_project, + resolve_project_path, +) + +INITIATIVE_ID_RE = re.compile( + r"^((?:QLIB|NQ|LAB|ORCH)-\d{3})\b", + re.IGNORECASE, +) +BRIEF_ID_RE = re.compile( + r"^((?:QLIB|NQ|LAB|ORCH)-\d{3})", + re.IGNORECASE, +) + +BOARD_TO_STATUS: dict[str, str] = { + "proposal": "paused", + "todo": "paused", + "plans": "in_progress", + "development": "in_progress", + "qa": "in_progress", + "done": "done", + "parked": "paused", +} + + +def now_iso() -> str: + return datetime.now(timezone.utc).replace(microsecond=0).isoformat() + + +def parse_initiative_id(title: str) -> str | None: + m = INITIATIVE_ID_RE.match(title.strip()) + return m.group(1).upper() if m else None + + +def gh_json(*args: str) -> Any: + result = subprocess.run( + ["gh", *args], + capture_output=True, + text=True, + check=False, + ) + if result.returncode != 0: + raise RuntimeError(f"gh {' '.join(args)} failed: {result.stderr.strip()}") + out = result.stdout.strip() + return json.loads(out) if out else {} + + +def fetch_board_items(project_number: int) -> list[dict[str, Any]]: + data = gh_json( + "project", + "item-list", + str(project_number), + "--owner", + GITHUB_LOGIN, + "--format", + "json", + "--limit", + "200", + ) + items: list[dict[str, Any]] = [] + for row in data.get("items", []): + content = row.get("content") or {} + title = row.get("title") or content.get("title") or "" + initiative_id = parse_initiative_id(title) + labels: list[str] = [] + repo = content.get("repository") or row.get("repository", "") + if isinstance(repo, str) and repo.startswith("http"): + repo = repo.rstrip("/").split("/")[-2] + "/" + repo.rstrip("/").split("/")[-1] + items.append( + { + "project_item_id": row.get("id"), + "board_status": row.get("status") or "Unknown", + "title": title, + "initiative_id": initiative_id, + "content_type": content.get("type"), + "issue_number": content.get("number"), + "url": content.get("url"), + "repo": repo if isinstance(repo, str) else "", + "state": content.get("state"), + } + ) + return items + + +def load_initiatives(path: Path) -> dict: + if not path.is_file(): + return {"version": 1, "updated_at": None, "initiatives": []} + return json.loads(path.read_text(encoding="utf-8")) + + +def save_initiatives(path: Path, data: dict) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + data["updated_at"] = now_iso() + path.write_text(json.dumps(data, indent=2) + "\n", encoding="utf-8") + + +def scan_portfolio_briefs(prefixes: list[str]) -> dict[str, dict]: + """Initiative id -> {title_slug, path} from tap-agent portfolio briefs.""" + if not PORTFOLIO_INITIATIVES_DIR.is_dir(): + return {} + prefix_set = {p.upper() for p in prefixes} + out: dict[str, dict] = {} + for brief in sorted(PORTFOLIO_INITIATIVES_DIR.glob("*.md")): + m = BRIEF_ID_RE.match(brief.name) + if not m: + continue + iid = m.group(1).upper() + prefix = iid.split("-")[0] + if prefix not in prefix_set: + continue + slug = brief.stem[len(iid) + 1 :] if brief.stem.startswith(iid) else brief.stem + title = slug.replace("-", " ").strip() or iid + out[iid] = {"path": str(brief), "title": title.title()} + return out + + +def board_index(items: list[dict]) -> dict[str, dict]: + by_id: dict[str, dict] = {} + for item in items: + iid = item.get("initiative_id") + if iid: + by_id[iid] = item + return by_id + + +def local_index(data: dict) -> dict[str, dict]: + return {i["id"]: i for i in data.get("initiatives", []) if i.get("id")} + + +def status_for_board_column(column: str) -> str: + return BOARD_TO_STATUS.get(column.strip().lower(), "paused") + + +def compare_project(project_id: str) -> dict[str, Any]: + project = get_project(project_id) + project_path = resolve_project_path(project) + initiatives_path = project_path / ".cursor/state/initiatives.json" + local = load_initiatives(initiatives_path) + local_by_id = local_index(local) + + board_items: list[dict] = [] + board_by_id: dict[str, dict] = {} + if project.get("project_number"): + board_items = fetch_board_items(int(project["project_number"])) + board_by_id = board_index(board_items) + + portfolio_by_id = scan_portfolio_briefs(project.get("prefixes", [project["prefix"]])) + + board_ids = set(board_by_id) + local_ids = set(local_by_id) + portfolio_ids = set(portfolio_by_id) + + return { + "project_id": project_id, + "project_path": str(project_path), + "initiatives_path": str(initiatives_path), + "board_items": board_items, + "board_by_id": board_by_id, + "local_by_id": local_by_id, + "portfolio_by_id": portfolio_by_id, + "board_only": sorted(board_ids - local_ids), + "local_only": sorted(local_ids - board_ids), + "portfolio_only": sorted(portfolio_ids - board_ids), + "board_and_portfolio": sorted(board_ids & portfolio_ids), + "drift": sorted( + iid + for iid in board_ids & local_ids + if local_by_id[iid].get("board_status") != board_by_id[iid].get("board_status") + ), + "untracked_board": [ + item for item in board_items if not item.get("initiative_id") + ], + } + + +def print_status(report: dict) -> None: + pid = report["project_id"] + print(f"\n{'=' * 60}") + print(f"Project: {pid}") + print(f" Path: {report['project_path']}") + print(f" State: {report['initiatives_path']}") + print( + f" Board: {len(report['board_items'])} items | " + f"Local: {len(report['local_by_id'])} initiatives | " + f"Portfolio briefs: {len(report['portfolio_by_id'])}" + ) + + if report["board_only"]: + print("\n On board, missing from initiatives.json:") + for iid in report["board_only"]: + b = report["board_by_id"][iid] + print(f" - {iid}: {b['board_status']} — {b['title'][:70]}") + + if report["local_only"]: + print("\n In initiatives.json, not on board:") + for iid in report["local_only"]: + loc = report["local_by_id"][iid] + print(f" - {iid}: status={loc.get('status')} — {loc.get('title', '')[:70]}") + + if report["portfolio_only"]: + print("\n Portfolio brief exists, no board card (board behind):") + for iid in report["portfolio_only"]: + p = report["portfolio_by_id"][iid] + print(f" - {iid}: {p['title']} ({p['path']})") + + if report["drift"]: + print("\n board_status drift (board vs local):") + for iid in report["drift"]: + b = report["board_by_id"][iid]["board_status"] + l = report["local_by_id"][iid].get("board_status", "(unset)") + print(f" - {iid}: board={b} local={l}") + + if report["untracked_board"]: + print("\n Board items without initiative id in title:") + for item in report["untracked_board"]: + print(f" - [{item['board_status']}] {item['title'][:70]}") + + if ( + not report["board_only"] + and not report["local_only"] + and not report["portfolio_only"] + and not report["drift"] + and not report["untracked_board"] + ): + print("\n ✓ In sync (board, local, portfolio briefs aligned for tracked ids).") + + if report["board_only"] or report["drift"]: + print(f"\n → Fix local: initiative-sync pull {pid} --apply") + + +def pull_project(project_id: str, apply: bool) -> int: + project = get_project(project_id) + if not project.get("project_number"): + print(f"Project {project_id} has no GitHub project board number.", file=sys.stderr) + return 1 + + report = compare_project(project_id) + path = Path(report["initiatives_path"]) + data = load_initiatives(path) + by_id = local_index(data) + changes = 0 + + for iid, board in report["board_by_id"].items(): + status = status_for_board_column(board["board_status"]) + github = None + if board.get("issue_number") and board.get("repo"): + repo = board["repo"] + if "/" not in repo and project.get("repo"): + repo = project["repo"] + github = { + "repo": repo, + "number": board["issue_number"], + "url": board.get("url"), + "project_item_id": board.get("project_item_id"), + } + + entry = by_id.get(iid) + if not entry: + entry = { + "id": iid, + "title": board["title"].split(":", 1)[-1].strip() if ":" in board["title"] else board["title"], + "status": status, + "board_status": board["board_status"], + "scope": f"Synced from GitHub board ({now_iso()[:10]})", + "started_at": None, + "last_update_at": now_iso(), + "completed_at": now_iso() if status == "done" else None, + "phases": [], + "metrics": { + "subagents_spawned": 0, + "files_changed": 0, + "build_runs": 0, + "test_runs": 0, + "prs_created": 0, + "prs_merged": 0, + "gate_pauses": 0, + }, + "blockers": [], + "sync": {"source": "board_pull", "at": now_iso()}, + } + if github: + entry["github"] = github + data.setdefault("initiatives", []).append(entry) + by_id[iid] = entry + changes += 1 + print(f" + add {iid} ({board['board_status']} → status {status})") + else: + patched = False + if entry.get("board_status") != board["board_status"]: + entry["board_status"] = board["board_status"] + patched = True + if entry.get("status") != status and status == "done": + entry["status"] = status + entry["completed_at"] = entry.get("completed_at") or now_iso() + patched = True + elif entry.get("status") not in ("done", "cancelled") and status != entry.get("status"): + entry["status"] = status + patched = True + if github and entry.get("github") != github: + entry["github"] = github + patched = True + entry["last_update_at"] = now_iso() + if patched: + changes += 1 + print(f" ~ update {iid} → board {board['board_status']}") + + if not apply: + if changes: + print(f"\nDry run: {changes} change(s). Re-run with --apply to write {path}") + else: + print("\nNothing to pull.") + return 0 + + if changes: + save_initiatives(path, data) + print(f"\nWrote {changes} change(s) to {path}") + else: + print("\nAlready up to date.") + return 0 + + +def plan_project(project_id: str) -> int: + project = get_project(project_id) + if not project.get("project_number"): + print(f"No board for {project_id}.", file=sys.stderr) + return 1 + report = compare_project(project_id) + missing = report["portfolio_only"] + if not missing: + print(f"\n{project_id}: all portfolio briefs have a board card.") + return 0 + print(f"\n{project_id}: create issues/cards for portfolio briefs missing on board:\n") + for iid in missing: + p = report["portfolio_by_id"][iid] + title = f"{iid}: {p['title']}" + print(f" gh issue create --repo {project['repo']} --title \"{title}\" --label proposal") + print(f" # then add to project #{project['project_number']} (Proposal column)\n") + return 0 + + +def main() -> int: + parser = argparse.ArgumentParser(description="Sync GitHub boards with initiatives.json") + sub = parser.add_subparsers(dest="command", required=True) + + p_status = sub.add_parser("status", help="Show drift (default)") + p_status.add_argument( + "--project", + "-p", + default="all", + help="qulib | notquality | tap-agent | all", + ) + + p_pull = sub.add_parser("pull", help="Pull board → initiatives.json") + p_pull.add_argument("project", help="qulib | notquality") + p_pull.add_argument("--apply", action="store_true", help="Write initiatives.json") + + p_plan = sub.add_parser("plan", help="List portfolio briefs missing from board") + p_plan.add_argument("project", help="qulib | notquality") + + args = parser.parse_args() + + if args.command == "status": + targets = [p["id"] for p in PROJECTS if p.get("project_number")] + if args.project != "all": + targets = [args.project] + exit_code = 0 + for pid in targets: + try: + print_status(compare_project(pid)) + except RuntimeError as exc: + print(f"\n{pid}: ERROR {exc}", file=sys.stderr) + exit_code = 1 + return exit_code + + if args.command == "pull": + return pull_project(args.project, apply=args.apply) + + if args.command == "plan": + return plan_project(args.project) + + return 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/project_registry.py b/scripts/project_registry.py new file mode 100644 index 0000000..d671a0b --- /dev/null +++ b/scripts/project_registry.py @@ -0,0 +1,59 @@ +"""Shared portfolio project registry for Tap-Agent automation scripts.""" + +from __future__ import annotations + +from pathlib import Path + +TAP_AGENT_ROOT = Path(__file__).resolve().parents[1] +PORTFOLIO_INITIATIVES_DIR = TAP_AGENT_ROOT / "portfolio" / "initiatives" + +# Default paths when ~/.tapagent/portfolio.json is missing (solo dev layout). +PROJECTS: list[dict] = [ + { + "id": "qulib", + "repo": "TapeshN/qulib", + "prefix": "QLIB", + "prefixes": ["QLIB"], + "project_number": 3, + "path": TAP_AGENT_ROOT.parent / "qulib", + }, + { + "id": "notquality", + "repo": "TapeshN/notquality-app", + "prefix": "NQ", + "prefixes": ["NQ", "LAB"], + "project_number": 4, + "path": TAP_AGENT_ROOT.parent / "notquality-app", + }, + { + "id": "tap-agent", + "repo": "TapeshN/Tap-Agent", + "prefix": "ORCH", + "prefixes": ["ORCH"], + "project_number": None, + "path": TAP_AGENT_ROOT, + }, +] + +GITHUB_LOGIN = "TapeshN" + + +def get_project(project_id: str) -> dict: + for p in PROJECTS: + if p["id"] == project_id: + return p + known = ", ".join(p["id"] for p in PROJECTS) + raise SystemExit(f"Unknown project {project_id!r}. Known: {known}") + + +def resolve_project_path(project: dict) -> Path: + """Prefer portfolio.json path when present.""" + portfolio_path = Path.home() / ".tapagent" / "portfolio.json" + if portfolio_path.is_file(): + import json + + data = json.loads(portfolio_path.read_text(encoding="utf-8")) + for entry in data.get("projects", []): + if entry.get("id") == project["id"] and entry.get("path"): + return Path(entry["path"]).expanduser() + return Path(project["path"]).expanduser() diff --git a/standards/skills/initiative-sync.md b/standards/skills/initiative-sync.md new file mode 100644 index 0000000..4aa6145 --- /dev/null +++ b/standards/skills/initiative-sync.md @@ -0,0 +1,46 @@ +--- +id: initiative-sync +title: /initiative-sync — Board ↔ initiatives.json +category: skill +priority: high +always_apply: false +targets: [cursor, claude, codex, copilot] +--- + +# /initiative-sync — Board ↔ initiatives.json + +Reconcile GitHub Project boards with `.cursor/state/initiatives.json` and tap-agent portfolio briefs. + +## Usage + +```bash +./bin/initiative-sync.sh status # all projects with boards +./bin/initiative-sync.sh status --project qulib +./bin/initiative-sync.sh pull qulib --apply # board → initiatives.json +./bin/initiative-sync.sh plan notquality # briefs missing from board +``` + +## What it checks + +| Source | Role | +|--------|------| +| **GitHub Project board** | Card column (`Proposal`, `Todo`, `Plans`, …) and issue/PR link | +| **initiatives.json** | Agent session state (phases, metrics, checkpoints) | +| **portfolio/initiatives/*.md** | Planned briefs in tap-agent (flags when board has no card) | + +## When to run + +- Session start (Conductor checklist) — after `gh project item-list`, run `status` +- After `/approve` or manual board moves — `pull --apply` +- When portfolio brief exists but no card — `plan ` then create issue + board item + +## Rules + +- **Pull** updates `board_status`, `github`, and `status` from the board; it does **not** delete local-only initiatives +- **Never** modifies another project's files when run from the wrong cwd — paths come from `scripts/project_registry.py` or `~/.tapagent/portfolio.json` +- Board titles must start with an initiative id (`QLIB-001`, `NQ-001`, …) to auto-link + +## Related + +- ORCH-013 proposal engine (creates `proposal` issues on board) +- [product-conductor.md](../rules/product-conductor.md) — board column lifecycle diff --git a/standards/skills/portfolio.md b/standards/skills/portfolio.md index 4d1dfba..cba0890 100644 --- a/standards/skills/portfolio.md +++ b/standards/skills/portfolio.md @@ -47,6 +47,8 @@ PORTFOLIO (5 projects, 3 active) Reads `~/.tapagent/portfolio.json` and each project's `initiatives.json`. +Board drift: run `./bin/initiative-sync.sh status` (see [initiative-sync.md](./initiative-sync.md)). + ## Rules - Cross-project reads only — never modify another project's state files