diff --git a/.github/workflows/build-and-push.yml b/.github/workflows/build-and-push.yml index bb7f080..78827ff 100644 --- a/.github/workflows/build-and-push.yml +++ b/.github/workflows/build-and-push.yml @@ -21,8 +21,10 @@ jobs: include: - image: mcp-server dockerfile: mcp_server/Dockerfile - - image: ui - dockerfile: ui/Dockerfile + - image: api + dockerfile: api/Dockerfile + - image: frontend + dockerfile: frontend/Dockerfile steps: - uses: actions/checkout@v4 diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 7124ac5..0ff1a71 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -36,9 +36,19 @@ jobs: python-version: '3.11' cache: pip - - name: Install dependencies + - uses: actions/setup-node@v4 + with: + node-version: '22' + + - name: Install Python dependencies run: pip install -r tests/requirements.txt + - name: Install frontend dependencies + run: | + cd frontend + corepack enable + yarn install --immutable + - name: Apply database migrations run: | for f in db/*.sql; do @@ -46,5 +56,12 @@ jobs: PGPASSWORD=prdforge psql -h localhost -U prdforge -d prdforge -f "$f" done - - name: Run tests + - name: Run Python tests run: python -m pytest tests/test_mcp_tools.py tests/test_ui_api.py -v --tb=short + + - name: Run frontend checks + run: | + cd frontend + yarn lint + yarn typecheck + yarn test --run diff --git a/AGENTS.md b/AGENTS.md index e0be825..d83a21c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -6,11 +6,12 @@ PRD Forge is a self-hosted sectional PRD management system. It stores documents ## Architecture -Three Docker Compose services: -- **PostgreSQL 16** (`postgres:16-alpine`) — 10 tables, 2 views, schema in `db/01_init.sql`, seed in `db/02_seed.sql` (SnapHabit sample, 12 sections), comments in `db/03_comments.sql`, replies+settings in `db/04_replies_and_settings.sql`, token stats in `db/05_token_stats.sql`, chat memory in `db/06_chat.sql` -- **MCP Server** (`mcp_server/server.py`, ~850 lines) — FastMCP with 31 tools, asyncpg, stdio + HTTP transports -- **Web UI** (`ui/app.py`, ~1637 lines) — FastAPI, dark theme with vertical nav rail, inline comments with replies, project settings, project creation/switching, force-directed dependency graph, and always-visible streaming Claude chat with selection-context injection -- **Shared** (`shared/settings.py`) — Settings schema + validation, imported by both MCP server and UI +Four Docker Compose services: +- **PostgreSQL 16** (`postgres:16-alpine`) — 11 tables, 2 views, schema in `db/01_init.sql`, seed in `db/02_seed.sql` (SnapHabit sample, 12 sections), comments in `db/03_comments.sql`, replies+settings in `db/04_replies_and_settings.sql`, token stats in `db/05_token_stats.sql`, chat memory in `db/06_chat.sql`, MCP activity in `db/08_mcp_activity.sql` +- **MCP Server** (`mcp_server/server.py`, ~1560 lines) — FastMCP with 31 tools, asyncpg, stdio + HTTP transports +- **Python API** (`api/app.py`, ~1620 lines) — FastAPI backend, REST endpoints for projects/sections/chat/comments/token-stats +- **Frontend** (`frontend/`, Next.js 15) — React 19, Tailwind v4, shadcn/ui. Proxies `/api/*` to Python API +- **Shared** (`shared/settings.py`) — Settings schema + validation, imported by both MCP server and Python API ## Key Design Principles @@ -30,27 +31,33 @@ Three Docker Compose services: | File | Purpose | ~Lines | |------|---------|--------| -| `docker-compose.yml` | 3-service stack definition | 68 | +| `docker-compose.yml` | 4-service stack definition | 75 | | `db/01_init.sql` | Schema DDL (tables, indexes, triggers, views) | 154 | | `db/02_seed.sql` | SnapHabit sample seed (12 sections, 12 deps) | 570 | | `db/05_token_stats.sql` | Token usage estimates table | 12 | | `db/06_chat.sql` | Chat tables migration (project chats + messages) | 41 | +| `db/08_mcp_activity.sql` | MCP write activity tracking | 12 | | `docs/tool-reference.md` | MCP tool table and usage examples | 100 | | `docs/data-model.md` | ER diagram, dependency types, statuses, tags | 141 | | `docs/scaling.md` | Multi-user and scaling guidance | 90 | | `db/03_comments.sql` | Inline comments table (section_comments) | 20 | | `db/04_replies_and_settings.sql` | Comment replies + project settings tables | 40 | | `shared/settings.py` | Settings schema, defaults, validation (shared) | 30 | -| `mcp_server/server.py` | MCP server with 31 tools | 1290 | -| `ui/app.py` | FastAPI web UI with nav rail, replies, settings, project creation/switching, always-visible streaming chat (selection context), light/dark theme | 1637 | -| `ui/static/fonts.css` | Vendored Google Fonts (@font-face declarations) | 200+ | -| `ui/static/fonts/` | Vendored woff2 font files (Inter + JetBrains Mono) | — | +| `mcp_server/server.py` | MCP server with 31 tools + activity tracking | 1560 | +| `api/app.py` | FastAPI Python API (REST endpoints, chat, auth) | 1620 | +| `api/auth.py` | Python auth middleware (session validation, role resolution) | 100 | +| `api/auth_contract.py` | Better Auth table/column contract verification | 50 | +| `api/errors.py` | Structured error responses (9 error codes) | 70 | +| `api/ws.py` | WebSocket token minting + verification (HMAC-SHA256) | 80 | +| `frontend/` | Next.js 15 frontend (React 19, Tailwind v4, shadcn/ui) | — | +| `frontend/server.ts` | Custom Node server with WS proxy | 45 | +| `frontend/prisma/schema.prisma` | Better Auth tables (7 models) | 110 | | `tests/conftest.py` | Test fixtures (db pool, cleanup, monkeypatch) | 64 | | `tests/test_mcp_tools.py` | MCP tool tests (53 tests) | 700+ | | `tests/test_ui_api.py` | UI endpoint tests (71 tests) | 940+ | | `tests/test_smoke.py` | CI smoke tests (MCP, DB, UI, seed data) | 85 | | `.github/workflows/test.yml` | CI: runs 124 tests on every PR to main | 50 | -| `.github/workflows/build-and-push.yml` | CD: builds Docker images on tag push | 63 | +| `.github/workflows/build-and-push.yml` | CD: builds 3 Docker images on tag push | 65 | ## MCP Tool Groups @@ -76,16 +83,22 @@ Three Docker Compose services: ## Database Schema Quick Reference -- **projects** — id, name, slug (unique), description, version, created_at, updated_at -- **sections** — id, project_id, parent_section_id, slug, title, section_type, sort_order, status, content, summary, tags[], notes, word_count (generated), created_at, updated_at. UNIQUE(project_id, slug), UNIQUE(project_id, id) -- **section_revisions** — id, section_id, revision_number, content, summary, change_description, created_at. UNIQUE(section_id, revision_number) +- **projects** — id, name, slug (unique), description, version, organization_id, created_by, created_at, updated_at +- **sections** — id, project_id, parent_section_id, slug, title, section_type, sort_order, status, content, summary, tags[], notes, word_count (generated), updated_by, created_at, updated_at. UNIQUE(project_id, slug), UNIQUE(project_id, id) +- **section_revisions** — id, section_id, revision_number, content, summary, change_description, created_by, created_at. UNIQUE(section_id, revision_number) - **section_dependencies** — id, project_id, section_id, depends_on_id, dependency_type, description. Composite FKs enforce same-project. UNIQUE(section_id, depends_on_id) -- **section_comments** — id, section_id, anchor_text, anchor_prefix, anchor_suffix, body, resolved, created_at, updated_at. Text anchoring uses prefix/suffix context (~40 chars each) for disambiguation. -- **comment_replies** — id, comment_id (FK section_comments), author ('user'|'claude' CHECK), body, created_at. Threaded replies on comments. -- **project_settings** — project_id (PK, FK projects), settings (JSONB, merged with defaults at read time), updated_at. Auto-trigger on update. -- **token_estimates** — id, project_id (FK projects), operation, full_doc_tokens, loaded_tokens, created_at. Tracks context savings per read operation. -- **project_chats** — id, project_id (unique FK projects), created_at, updated_at. One chat thread per project. -- **chat_messages** — id, chat_id (FK project_chats), role ('user'|'assistant'|'system'|'tool' CHECK), content, metadata (JSONB), created_at. +- **section_comments** — id, section_id, anchor_text, anchor_prefix, anchor_suffix, body, resolved, created_by, created_at, updated_at +- **comment_replies** — id, comment_id (FK section_comments), author ('user'|'claude' CHECK), body, created_at +- **project_settings** — project_id (PK, FK projects), settings (JSONB, merged with defaults at read time), updated_at +- **token_estimates** — id, project_id (FK projects), operation, full_doc_tokens, loaded_tokens, created_at +- **project_chats** — id, project_id, chat_type ('main'|section), section_id, created_by, created_at, updated_at. Multi-thread support. +- **chat_messages** — id, chat_id (FK project_chats), role, content, metadata (JSONB), created_by, created_at +- **mcp_activity** — id, project_id, tool_name, detail (JSONB), user_id, created_at. 12 mutating tools. +- **project_members** — id, project_id, user_id, role (owner/admin/editor/commenter/viewer), created_at, updated_at +- **audit_events** — id, project_id, user_id, action, resource, detail (JSONB), created_at +- **password_reset_tokens** — id, user_id, token, expires_at, used, created_by, created_at +- **prdforge_bootstrap** — id, setup_type (unique), completed, created_at +- Better Auth tables: user, session, account, verification, organization, member, invitation (managed by Prisma) - **section_tree** (view) — sections + project_slug, parent_slug, parent_title, revision_count, dep_out_count, dep_in_count - **project_changelog** (view) — revisions joined with section and project slugs @@ -112,8 +125,8 @@ Three Docker Compose services: 3. Update `db/02_seed.sql` if the seed data format changed 4. Update AGENTS.md schema reference -**Adding a UI endpoint:** -1. Add the route to `ui/app.py` +**Adding an API endpoint:** +1. Add the route to `api/app.py` 2. Query the pool directly 3. Add a test in `test_ui_api.py` @@ -158,20 +171,13 @@ python -m venv .venv && .venv/bin/pip install -r tests/requirements.txt - **Chat is experimental** — disabled by default, gated behind `chat_enabled` project setting. All 4 chat endpoints return 403 when disabled. Enable in Settings → Experimental Features. - **Chat model selector** — `chat_model` setting (`sonnet`/`opus`/`haiku`) per-project, stored in `project_settings` JSONB. Passed to CLI as `--model` flag. For API provider, mapped via `API_MODEL_MAP` dict. - **Section status editor** — `PATCH /api/projects/{slug}/sections/{section}` supports updating status, tags, title, summary. Valid statuses: `draft`, `in_progress`, `review`, `approved`, `outdated`. -- Web UI Claude chat supports two auth methods: Claude CLI OAuth login (credentials file) or Anthropic API key +- Web UI Claude chat uses Anthropic API key for authentication - Chat tool execution uses an allowlist of MCP tool functions with project slug enforced server-side - Web UI chat can attach selected section text as context; backend stores this in `chat_messages.metadata.selection_context` and rehydrates it into future model history turns - Web UI chat can attach local files (text payloads); backend stores them in `chat_messages.metadata.attachments` and injects their content into future model history turns -- Web UI chat provider defaults to `CHAT_PROVIDER` (`claude_cli` or `anthropic_api`) and can be overridden per project via settings (`chat_provider`). In Docker CLI mode, UI container needs `claude` binary plus mounted auth dir (`${HOME}/.claude` → `/root/.claude`) +- Web UI chat provider can be overridden per project via settings (`chat_provider`) - Web UI chat renders selected context inline inside user message bubbles and triggers best-effort live refresh of project/section views after each completed assistant turn - Chat attachment limits are controlled via env vars: `CHAT_MAX_ATTACHMENTS`, `CHAT_ATTACHMENT_MAX_BYTES`, `CHAT_ATTACHMENT_MAX_CHARS`, `CHAT_ATTACHMENTS_MAX_TOTAL_CHARS` -- **Claude CLI auth in Docker:** The CLI reads credentials from `/root/.claude/.credentials.json` (written by the OAuth login flow). Do NOT pass `ANTHROPIC_AUTH_TOKEN` as env var to the CLI subprocess — it causes 401 "OAuth authentication is currently not supported" because the CLI tries to use the token as a Bearer header against the Messages API, which rejects OAuth tokens. The credentials file approach works because the CLI uses its own internal auth flow. -- **OAuth token format:** The callback page returns `AUTH_CODE#STATE` — split on `#` to get the code and state separately. -- **OAuth tokens vs API keys:** OAuth tokens (`sk-ant-oat01-*`) do NOT work with the Anthropic Messages API. They only work through the Claude CLI which routes via a different backend. For direct API calls, a real API key (`sk-ant-api03-*`) is required. -- **CLI default model:** The CLI's built-in default model may be deprecated. Always pass `--model sonnet` (or user-selected model) explicitly. Never rely on the CLI default. -- **CLI `--allowedTools` required:** In non-interactive mode (`-p` flag), the CLI cannot prompt for tool approval. MCP tools will silently fail unless `--allowedTools` is passed with the explicit list of allowed tool names. -- In `claude_cli` mode with manual permission settings (e.g. `acceptEdits`), permission requests are surfaced as dedicated `approval` chat events and persisted in `chat_messages.metadata.approval_requests` -- Approval cards in Web UI chat support **Approve and continue** via `/api/projects/{slug}/chat/approve`, which replays the blocked user turn with `CLAUDE_CLI_APPROVAL_PERMISSION_MODE` (default `acceptEdits`; legacy `dontAsk` is normalized), passes a strict PRD-tool allowlist to Claude CLI, and marks the original approval message as resolved in metadata - `GET /api/projects/{slug}` now backfills missing initial revisions for sections; if a project has chat activity and zero dependencies, it backfills a linear references chain so Dependencies/Changelog tabs are populated for chat-generated projects - `GET /api/projects/{slug}/token-stats` now includes `project_stats` (`sections`, `dependencies`, `revisions`) in addition to token-savings metrics - `install.sh` now auto-selects a free host PostgreSQL port (`5432`, else first free in `5433-5500`) and exports `POSTGRES_PORT` so Docker + Claude Desktop config stay in sync diff --git a/README.md b/README.md index 004dca1..eb58370 100644 --- a/README.md +++ b/README.md @@ -1,14 +1,14 @@ # PRD Forge -![PRDforge Demo](demo.gif) - **Stop feeding your entire spec to Claude every time you change one paragraph.** +![PRDforge Demo](demo.gif) + PRD Forge splits your product requirements into independently addressable sections stored in PostgreSQL, then gives Claude surgical read/write access through 31 MCP tools. The result: edits that used to burn ~15,000 tokens now cost 500-2,000 — an **85-95% reduction** in context per operation. ## The Problem -Every AI-assisted PRD workflow today has the same bottleneck: Claude needs the full document loaded into context to make a single edit. A 20-page spec means ~15K tokens of context consumed on every interaction — even if you're only changing one section. That context cost adds up fast, limits how much Claude can reason about, and makes large specs unwieldy. +Every AI-assisted PRD workflow today has the same bottleneck: Claude needs the full document loaded into context to make a single edit. A 20-page spec means ~15K tokens of context consumed on every interaction — even if you're only changing one section. ## How PRD Forge Solves It @@ -20,16 +20,20 @@ Each section stores both its full **content** and a short **summary** (1-3 sente **Real example:** Reading `data-model` (820 words, ~1,200 tokens) loads summaries of `tech-stack` (~60 tokens) and `pipeline` (~60 tokens). Total: **~1,320 tokens** instead of ~15,000. -Claude always has enough context to make informed edits without paying for the entire document. - -## What You Get - -- **31 MCP tools** — read, write, search, import/export, manage dependencies, track revisions, resolve comments. Claude operates on your spec like a database, not a blob of text. -- **Dependency-aware context** — sections know what they depend on. When Claude reads one, it automatically gets summaries of upstream sections for context. -- **Full revision history** — every content change creates a revision. Roll back any section to any point. No content is ever lost. -- **Google Docs-style comments** — leave inline comments anchored to specific text, Claude reads them, implements changes, resolves them. Threaded replies included. -- **Web UI** — browse specs, leave comments, create new projects, view dependency graph, toggle dark/light theme. Experimental: always-visible project Claude chat with selection context, file attachments, and model selector. -- **One command to install** — `./install.sh` handles Docker, MCP config, and validation in ~15 seconds. +## Features + +- **31 MCP tools** — read, write, search, import/export, manage dependencies, track revisions, resolve comments +- **Multi-user auth** — Better Auth (email/password + Google OAuth), 5 roles (owner/admin/editor/commenter/viewer), org-scoped access control +- **Real-time collaboration** — WebSocket presence, live section updates across clients +- **Project templates** — start with Blank, SaaS MVP, Mobile App, or API Design — pre-built section structures with starter content +- **Dependency-aware context** — sections know what they depend on; Claude automatically gets upstream summaries +- **Full revision history** — every content change creates a revision, roll back to any point +- **Google Docs-style comments** — inline comments anchored to specific text, threaded replies +- **AI chat** — streaming chat panel with Claude (Anthropic API or CLI), tool-calling into your PRD +- **Dependency graph** — interactive SVG visualization of section relationships +- **Token stats** — dashboard showing per-section token usage +- **Next.js frontend** — React 19, Tailwind v4, shadcn/ui, dark/light theme +- **One command install** — `./install.sh` handles Docker, MCP config, validation ## Architecture @@ -37,14 +41,21 @@ Claude always has enough context to make informed edits without paying for the e graph LR A[Claude.ai / Claude Code / Claude Desktop] <-->|MCP Protocol| B[MCP Server
FastMCP/Python
:8080] B <-->|asyncpg| C[(PostgreSQL 16
sections, revisions
dependencies, comments)] - D[Web UI
FastAPI
:8088] <-->|read + comments + chat memory| C + D[Python API
FastAPI
:8088] <-->|read + write + chat| C + E[Frontend
Next.js 15
:3000] <-->|REST proxy| D + F[Redis 7] <-->|pub/sub, jti| D A -.->|reads comments| D ``` -Three Docker services, all localhost-only: -- **PostgreSQL 16** — source of truth (10 tables, 2 views) -- **MCP Server** — 31 tools for Claude integration (stdio + HTTP transports) -- **Web UI** — browser interface with project switching/creation, inline comments, dependency graph, dark/light theme. Experimental chat with model selector and MCP tool access +Five Docker services: + +| Service | Stack | Port | Purpose | +|---------|-------|------|---------| +| **PostgreSQL 16** | 15+ tables, 2 views | 5432 | Source of truth | +| **MCP Server** | FastMCP / Python | 8080 | 31 tools for Claude | +| **Python API** | FastAPI | 8088 | REST backend (projects, sections, chat, auth, audit) | +| **Frontend** | Next.js 15, React 19, Tailwind v4, shadcn/ui, Better Auth | 3000 | Web UI | +| **Redis 7** | — | 6379 | WebSocket token uniqueness, real-time pub/sub | ## Quick Start @@ -55,7 +66,7 @@ cd PRDforge This single command: 1. Pulls pre-built images from ghcr.io (or builds locally if unavailable) -2. Starts Docker services (PostgreSQL, MCP server, Web UI) +2. Starts Docker services (PostgreSQL, MCP server, API, Frontend, Redis) 3. Configures your Claude client (Code or Desktop) 4. Validates everything works @@ -68,17 +79,15 @@ This single command: POSTGRES_PORT=5433 ./install.sh # Override host PostgreSQL port ``` -If `5432` is already in use, `install.sh` automatically picks the first free port in `5433-5500` and configures it for Docker + Claude Desktop. +If `5432` is already in use, `install.sh` automatically picks the first free port in `5433-5500`. -For Claude Desktop setup, `install.sh` now auto-selects a compatible Python interpreter (`3.10-3.13`) for `mcp_server/.venv` and recreates that venv if it was previously created with an unsupported Python (for example, `3.14`). +The stack starts in ~15 seconds. PostgreSQL seeds a sample "SnapHabit" project (12 sections, 12 dependencies) on first boot. -The stack starts in ~15 seconds. Chat is disabled by default — enable it per-project in **Settings → Experimental Features**. PostgreSQL seeds a sample "SnapHabit" project (12 sections, 12 dependencies) on first boot — a mobile habit-tracking app with AWS serverless backend. Edit or delete the seed data to start your own PRD. +After install, restart your Claude client. Web UI: http://localhost:3000 -After install, restart your Claude client. Web UI: http://localhost:8088 +## Configuration -## MCP Configuration (Manual) - -If you prefer to configure manually instead of using `install.sh`: +### MCP Configuration (Manual)
Claude Code (HTTP — recommended with Docker) @@ -126,7 +135,6 @@ Start services: `docker compose up -d` 4. Restart Claude Desktop (Cmd+Q, reopen) > **Note:** Claude Desktop does not support HTTP transport. Use stdio (spawns server as subprocess). -> If PostgreSQL is published on a non-default host port, replace `5432` in `DATABASE_URL` with your chosen port.
@@ -144,7 +152,18 @@ Start services: `docker compose up -d` ```
-## Tool Reference +### Chat Configuration + +Chat is an experimental feature, disabled by default. Enable per-project in **Settings → Experimental Features**. + +| Variable | Default | Description | +|----------|---------|-------------| +| `ANTHROPIC_API_KEY` | — | Anthropic API key for chat | +| `ANTHROPIC_MODEL` | `claude-haiku-4-5-20251001` | Model for chat responses | +| `CHAT_MAX_ATTACHMENTS` | `5` | Max files per chat turn | +| `CHAT_ATTACHMENT_MAX_BYTES` | `200000` | Max size per file payload | + +## MCP Tools Reference 31 MCP tools across 10 groups: project management, section CRUD, dependencies, comments, context/search, revisions, import/export, batch operations, token stats, and settings. @@ -154,58 +173,65 @@ See **[docs/tool-reference.md](docs/tool-reference.md)** for the full tool table Google Docs-style comments anchored to specific text in any section: -1. **In the UI** — select text in a section → click "+ Comment" → write your note → Save -2. **Via MCP** — `prd_add_comment(project, section, anchor_text, body)` with optional `anchor_prefix`/`anchor_suffix` for disambiguation -3. **Claude scans comments** — `prd_list_comments` returns all open comments with section pointers (~100 tokens), then read only the sections that have feedback -4. **Claude reads comments** — `prd_read_section` includes all comments with their anchor text and body -5. **Resolve after implementing** — use `prd_resolve_comment` or click "Resolve" in the UI - -Workflow: leave comments → ask Claude to check feedback → Claude calls `prd_list_comments` to see which sections have comments → reads only those sections → implements changes → resolves comments. - -## Web UI Claude Chat (Experimental) +1. **In the UI** — select text → click "+ Comment" → write your note → Save +2. **Via MCP** — `prd_add_comment(project, section, anchor_text, body)` +3. **Claude scans comments** — `prd_list_comments` returns all open comments +4. **Resolve after implementing** — `prd_resolve_comment` or click "Resolve" in the UI -Chat is an **experimental feature**, disabled by default. Enable it per-project in **Settings → Experimental Features → Chat Integration**. +## Development -### Authentication +```bash +# Run backend tests (requires postgres running) +docker compose up -d postgres +pip install -r tests/requirements.txt +pytest tests/ -x -v -Chat requires an Anthropic API key. Paste your `sk-ant-api03-...` key in **Settings → Experimental Features → Anthropic API Key** and set provider to "Anthropic API". +# Frontend type checking and lint +cd frontend && yarn typecheck && yarn lint && yarn test --run -### Features +# Smoke tests (requires full stack) +docker compose up -d +pytest tests/test_smoke.py -v -- One persistent thread per project, stored in PostgreSQL (`project_chats`, `chat_messages`) -- Streaming responses in the browser (`text/event-stream`) -- **Model selector** — choose between Sonnet 4.6, Opus 4.6, and Haiku 4.5 per-project -- **Selection context** — highlight text in a section and your next chat message includes that exact snippet -- **File attachments** in chat composer (name/type/size + text content) -- After each assistant turn, the UI live-refreshes project/section data to reflect agent-applied updates -- Chat has access to PRDforge MCP tools — it can read, search, and edit your PRD sections directly -- In Claude CLI mode, approval-required tool requests are shown inline with an **Approve and continue** action -- Default provider uses local `claude` command in the UI container (`CHAT_PROVIDER=claude_cli`) -- API-key fallback available (`CHAT_PROVIDER=anthropic_api` + `ANTHROPIC_API_KEY`) -- Provider and model can be overridden per project in **Settings → Experimental Features** +# Record demo video +pip install -r scripts/requirements.txt +playwright install chromium +python scripts/record_demo.py +``` -### Chat Env Variables +**Project structure:** +``` +PRDforge/ +├── docker-compose.yml +├── frontend/ # Next.js 15 app (React 19, Tailwind v4, shadcn/ui) +├── api/ # FastAPI backend (REST, chat, auth, WebSocket) +├── mcp_server/ # FastMCP server (31 tools, stdio + HTTP) +├── shared/ # Shared Python modules (settings, constants, templates) +├── db/ # PostgreSQL schema migrations (13 files) +├── tests/ # pytest test suite +├── scripts/ # Demo recording, utilities +└── docs/ # Tool reference, data model, scaling guide +``` -- `CHAT_PROVIDER=claude_cli` — run chat via terminal command `claude` (default) -- `CHAT_PROVIDER=anthropic_api` — use direct Anthropic API (`ANTHROPIC_API_KEY` required) -- `CLAUDE_CLI_PATH` — command path override (default `claude`) -- `CLAUDE_CLI_MODEL` — default model (default `sonnet`; overridden by per-project setting) -- `CLAUDE_CLI_APPROVAL_PERMISSION_MODE` — permission mode for **Approve and continue** (default `acceptEdits`) -- `CLAUDE_CLI_ARGS` — optional extra CLI args -- `CHAT_MAX_ATTACHMENTS` — max files per chat turn (default `5`) -- `CHAT_ATTACHMENT_MAX_BYTES` — max size per file payload (default `200000`) -- `CHAT_ATTACHMENT_MAX_CHARS` — max extracted text per file (default `12000`) -- `CHAT_ATTACHMENTS_MAX_TOTAL_CHARS` — max combined attachment text per turn (default `40000`) +## Security & Deployment -## Data Model & Reference +PRD Forge ships with **Better Auth** (email/password + Google OAuth) and role-based access control. Sign-up is closed after the first user is created — new users are invited via organization member management. -10 tables, 2 views. See **[docs/data-model.md](docs/data-model.md)** for the full ER diagram, dependency types, tags, statuses, and the SnapHabit example dependency graph. +**Localhost-only by default:** +- All ports bound to `127.0.0.1` — not accessible from LAN +- No TLS — acceptable only for localhost +- Database credentials are defaults (`prdforge`/`prdforge`) -## Seed Data +**For production deployment:** +- Put behind a reverse proxy with TLS (nginx, Caddy, Traefik) +- Change database credentials in `.env` +- Set `BETTER_AUTH_SECRET` to a strong random value +- See [docs/scaling.md](docs/scaling.md) for detailed guidance +- **Do NOT** bind ports to `0.0.0.0` or expose via tunnels without auth -The default seed (`db/02_seed.sql`) creates a "SnapHabit" project — a mobile habit-tracking app with AWS serverless backend. It has 12 sections (overview, user research, tech stack, data model, API spec, mobile app, push notifications, auth, deployment, analytics, testing strategy, timeline) and 12 dependencies. +## Data Model -Edit or delete these to start your own PRD. To start fresh, simply clear the seed and add your own sections. +15+ tables, 2 views. See **[docs/data-model.md](docs/data-model.md)** for the full ER diagram, dependency types, tags, statuses, and the SnapHabit example. ## Backup & Restore @@ -217,87 +243,7 @@ curl http://localhost:8088/api/projects/snaphabit/export > backup.md docker exec prdforge-postgres-1 pg_dump -U prdforge prdforge > backup.sql # Full reset (destroys all data) -docker compose down -v -docker compose up -d -``` - -## Security Notes - -> **WARNING: PRD Forge is a single-user local-only tool.** It has no authentication, no authorization, and no rate limiting. It is NOT suitable for team use, shared access, or deployment on a network. - -- All ports bound to `127.0.0.1` — not accessible from LAN by default -- **Do NOT** bind ports to `0.0.0.0`, expose them via tunnels (ngrok, Cloudflare Tunnel), or open firewall rules. Anyone with network access to port 8080 gets full read/write access to all your data with zero authentication. -- Database credentials are defaults (`prdforge`/`prdforge`) — acceptable only for localhost -- If you must expose PRD Forge beyond localhost, put it behind a reverse proxy with TLS and authentication. See [docs/scaling.md](docs/scaling.md) for guidance. - -## Known Limitations - -- No latency/error-rate metrics beyond structured logging, `/health`, and `prd_token_stats` -- No reverse proxy hardening — localhost-only binding prevents accidental exposure - -## Development - -```bash -# Run unit tests (requires postgres running) -docker compose up -d postgres -pip install -r tests/requirements.txt -pytest tests/test_mcp_tools.py tests/test_ui_api.py -v - -# Smoke tests (requires full stack) -docker compose up -d -pip install httpx pytest -pytest tests/test_smoke.py -v -``` - -**Project structure:** -``` -PRDforge/ -├── docker-compose.yml -├── docker-compose.prod.yml -├── .env.example -├── .gitignore -├── .github/workflows/build-and-push.yml -├── claude_mcp_config.json -├── README.md -├── AGENTS.md -├── prd.md -├── docs/ -│ ├── tool-reference.md -│ ├── data-model.md -│ └── scaling.md -├── db/ -│ ├── 01_init.sql -│ ├── 02_seed.sql -│ ├── 03_comments.sql -│ ├── 04_replies_and_settings.sql -│ ├── 05_token_stats.sql -│ └── 06_chat.sql -├── shared/ -│ ├── __init__.py -│ └── settings.py -├── mcp_server/ -│ ├── Dockerfile -│ ├── requirements.txt -│ └── server.py -├── ui/ -│ ├── Dockerfile -│ ├── requirements.txt -│ ├── app.py -│ └── static/ -│ ├── marked.min.js -│ ├── highlight.min.js -│ ├── github-dark.min.css -│ ├── fonts.css -│ ├── fonts/ -│ │ ├── inter-*.woff2 -│ │ └── jetbrains-mono-*.woff2 -│ └── MARKED_VERSION -└── tests/ - ├── requirements.txt - ├── conftest.py - ├── test_mcp_tools.py - ├── test_ui_api.py - └── test_smoke.py +docker compose down -v && docker compose up -d ``` ## License diff --git a/TODO.md b/TODO.md index 5ee5ba4..c0983a6 100644 --- a/TODO.md +++ b/TODO.md @@ -2,43 +2,42 @@ ## High Impact, Moderate Effort – Do Next -- [x] Demo GIF/video in README — Playwright-based recording script (`scripts/record_demo.py`), GIF above the fold -- [ ] Token savings dashboard in Web UI — chart showing cumulative tokens saved over time, savings per section read, comparison bar ("full doc: 15,000 tokens vs loaded: 1,320 tokens"). Proof-of-value screen users will screenshot and share -- [ ] `prd_import_url` tool — fetch a Google Doc, Notion page, or raw URL and import as sections. Support public Google Docs and raw GitHub markdown files to reduce onboarding friction -- [ ] Section templates — offer project templates beyond blank/SnapHabit: "SaaS MVP", "API Design", "Mobile App", "Infrastructure Migration". Each is a seed SQL or importable markdown showcasing different dependency patterns -- [ ] move to github projects -- [ ] add playwrite preview autoupdate (ci for pr + agents.md for agents instaction which sections observe) +- [ ] Agent personas for chat — per-project `system_prompt` in settings + per-section `agent_prompt` override. Ship 5-6 presets (PRD Architect, Technical Reviewer, UX/Design, QA Strategist, Executive Summary). Users pick preset or write custom. Passed as system message to the chat provider. Section-level prompt overrides project-level when chatting in that section context. +- [ ] Section templates — offer project templates beyond blank/SnapHabit: "SaaS MVP", "API Design", "Mobile App", "Infrastructure Migration". Each is a seed SQL or importable markdown showcasing different dependency patterns. +- [ ] add notes (accordion) to section. Here, I would like the option to add a note to the entire section. +- [ ] We need to update the readme relative to all our recent changes, create a preview as we did through the playwrite, and make it as commercially product-successful as possible to attract potential users' attention. ## Medium Impact, Lower Effort – Quick Wins -- [ ] Health check in docker-compose — add `healthcheck` directives for MCP server and UI containers (UI already has `/health`). Fixes startup ordering, makes `docker compose up` more reliable - [ ] CONTRIBUTING.md — how to run tests locally, code style expectations, PR process - [ ] GitHub repo metadata — description, topics/tags (`mcp`, `claude`, `prd`, `product-requirements`, `ai-tools`, `developer-tools`, `mcp-server`), website URL -- [ ] Multi-arch Docker builds — add `linux/amd64,linux/arm64` to GH Actions build for Apple Silicon support - [ ] `prd_diff_sections` tool — unified diff between two revisions of a section, avoids loading both and diffing manually -- [ ] ui playwrite tests +- [ ] ui playwrite tests +- [ ] add playwrite preview autoupdate (ci for pr + agents.md for agents instaction which sections observe) ## Medium Impact, Higher Effort – Plan For These - [ ] Webhooks / notifications — POST to a configured URL on section update for Slack/Discord/pipeline integration - [ ] Export to Google Docs / Notion — push assembled PRD back to where stakeholders read it (Notion API is more reasonable than Google Docs) -- [ ] Section-level permissions / locking — advisory lock with timestamp and owner field so two Claude sessions don't clobber each other - [ ] Embeddings-based dependency suggestions — store embeddings in pgvector, compute cosine similarity to catch semantic relationships that FTS keyword overlap misses - [ ] Version tagging / snapshots — `prd_tag_version` tool that snapshots all current section revision numbers under a named tag for milestone tracking +- [ ] Google OAuth provider — Better Auth config ready, needs Google Cloud Console credentials ## Lower Priority – Worth Tracking -- [ ] Conflict detection — optimistic concurrency with revision check on write (warn/fail if section was edited since last read) +- [ ] `prd_import_url` tool — fetch public Google Docs, GitHub markdown, or raw URLs and import as sections. Private docs deferred (each provider needs its own OAuth — not worth the complexity for an import tool). Public-only version is ~2h of work. - [ ] Section ordering visualization — Gantt-style view based on `blocks` dependency type for timeline sections - [ ] Prometheus `/metrics` endpoint — request counts, latency histograms, DB connection pool stats - [ ] claude.ai MCP marketplace listing — be ready when Anthropic opens MCP server discovery - [ ] Replace heuristic markdown import parser with `markdown-it-py` AST parser -- [ ] Add WebSocket push to UI for real-time updates when sections change - [ ] Support `### ` (h3) splitting in import for nested section hierarchies - [ ] Add `prd_merge_sections` tool (combine two sections into one) - [ ] Add `prd_reorder_sections` tool (bulk sort_order update) - [ ] Export as PDF via headless browser - [ ] UI: keyboard navigation (j/k to move through sections, Enter to select) +- [ ] MCP auth for remote Claude clients (SSH tunnel or authenticated ingress) +- [ ] Redis jti uniqueness for WS tokens (SET NX EX — currently TODO in code) +- [ ] move to tasks github projects ## Done @@ -68,3 +67,24 @@ - [x] Google Fonts vendored locally (Inter, JetBrains Mono in ui/static/fonts/) - [x] Light theme / dark-light toggle in UI nav rail - [x] Docker build pipeline (GitHub Actions → ghcr.io) + `docker-compose.prod.yml` + install.sh pull support +- [x] Token savings dashboard — Grafana-style with recharts, area/bar/pie/gauge, session-based honest savings math, section heatmap +- [x] Health checks in docker-compose — all services (postgres, mcp-server, redis) have healthchecks with depends_on conditions +- [x] Multi-arch Docker builds — `linux/amd64,linux/arm64` in GH Actions +- [x] Multi-user auth — Better Auth (email/password), closed sign-up, first-user bootstrap +- [x] RBAC — project_members with 5 roles (owner/admin/editor/commenter/viewer), permission middleware +- [x] Next.js frontend — React 19, Tailwind v4, shadcn/ui, replaces server-rendered HTML +- [x] Real-time WebSocket — presence tracking, event broadcasting via Redis pub/sub +- [x] Optimistic locking — expected_revision on section PATCH, 409 Conflict with details +- [x] MCP activity tracking — 12 mutating tools logged to mcp_activity table +- [x] Chat with streaming — SSE, tool calls display, selection context, file attachments, stop button +- [x] Audit events table — project + user indexed +- [x] Password reset flow — admin-generated tokens, no email +- [x] Member management — add/remove/change role via API + UI +- [x] Dependency graph — dual view (force-directed SVG graph + list with type badges), click popup, drag nodes, curved colored arrows +- [x] Structured error handling — 9 error codes, frontend error boundaries +- [x] Comments — text selection anchoring, resolve/reopen toggle, edit/delete, existing comment detection +- [x] Section status management — clickable dropdown to change status +- [x] Multi-user chat threads — per-project + per-section, chat_type column +- [x] Org-level encrypted API key — Fernet/AES-256 +- [x] Wider chat section — 40% viewport width, min 480px, max 700px +- [x] Honest token savings math — section_access_log with session-based dedup, coverage fractions (full/summary/snippet), 30-min session windowing diff --git a/ui/Dockerfile b/api/Dockerfile similarity index 62% rename from ui/Dockerfile rename to api/Dockerfile index 621698b..838e400 100644 --- a/ui/Dockerfile +++ b/api/Dockerfile @@ -5,11 +5,12 @@ RUN apt-get update \ && npm install -g @anthropic-ai/claude-code \ && apt-get clean \ && rm -rf /var/lib/apt/lists/* -COPY ui/requirements.txt /app/ui/ -RUN pip install --no-cache-dir -r /app/ui/requirements.txt +COPY api/requirements.txt /app/api/ +RUN pip install --no-cache-dir -r /app/api/requirements.txt COPY shared/ /app/shared/ COPY mcp_server/ /app/mcp_server/ -COPY ui/app.py /app/ui/ -COPY ui/static/ /app/ui/static/ -WORKDIR /app/ui +COPY api/app.py api/auth.py api/errors.py api/ws.py /app/api/ +COPY api/auth_contract.py /app/api/ +COPY api/static/ /app/api/static/ +WORKDIR /app/api CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8088"] diff --git a/ui/app.py b/api/app.py similarity index 73% rename from ui/app.py rename to api/app.py index 3bf45c5..40ddb16 100644 --- a/ui/app.py +++ b/api/app.py @@ -17,13 +17,14 @@ import asyncpg import httpx -from fastapi import FastAPI, Request -from fastapi.responses import HTMLResponse, JSONResponse, PlainTextResponse, StreamingResponse -from fastapi.staticfiles import StaticFiles +from fastapi import FastAPI, Request, WebSocket, WebSocketDisconnect +from fastapi.responses import JSONResponse, PlainTextResponse, StreamingResponse sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..")) sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "mcp_server")) from shared.settings import CHAT_PROVIDER_VALUES, DEFAULT_PROJECT_SETTINGS, validate_settings +from shared.project_factory import create_project_with_template +from shared.templates import list_templates pool: asyncpg.Pool | None = None logger = logging.getLogger("prd_forge_ui") @@ -73,17 +74,31 @@ def _valid_project_slug(slug: str) -> bool: return bool(slug and len(slug) <= 100 and PROJECT_SLUG_RE.match(slug)) +redis_client = None + + @asynccontextmanager async def lifespan(app: FastAPI): - global pool + global pool, redis_client pool = await asyncpg.create_pool(os.environ["DATABASE_URL"]) + redis_url = os.environ.get("REDIS_URL", "") + if redis_url: + try: + import redis.asyncio as aioredis + redis_client = aioredis.from_url(redis_url, decode_responses=True) + await redis_client.ping() + logger.info("Redis connected: %s", redis_url) + except Exception as e: + logger.warning("Redis not available (real-time features disabled): %s", e) + redis_client = None yield + if redis_client: + await redis_client.aclose() if pool: await pool.close() -app = FastAPI(title="PRD Forge UI", lifespan=lifespan) -app.mount("/static", StaticFiles(directory=os.path.join(os.path.dirname(__file__), "static")), name="static") +app = FastAPI(title="PRD Forge API", lifespan=lifespan) def dt(v): @@ -108,9 +123,6 @@ def row_dict(r): return d -_static_dir = os.path.join(os.path.dirname(__file__), "static") -with open(os.path.join(_static_dir, "index.html")) as _f: - HTML = _f.read() CHAT_ALLOWED_MCP_TOOLS: dict[str, dict[str, Any]] = { @@ -228,6 +240,237 @@ def row_dict(r): }, "arg_names": ["section", "comment_id", "body"], }, + "prd_add_comment": { + "description": "Add inline comment anchored to text in a section.", + "input_schema": { + "type": "object", + "properties": { + "section": {"type": "string"}, + "anchor_text": {"type": "string"}, + "body": {"type": "string"}, + "anchor_prefix": {"type": "string"}, + "anchor_suffix": {"type": "string"}, + }, + "required": ["section", "anchor_text", "body"], + "additionalProperties": False, + }, + "arg_names": ["section", "anchor_text", "body", "anchor_prefix", "anchor_suffix"], + }, + "prd_delete_comment": { + "description": "Delete an inline comment.", + "input_schema": { + "type": "object", + "properties": { + "section": {"type": "string"}, + "comment_id": {"type": "string"}, + }, + "required": ["section", "comment_id"], + "additionalProperties": False, + }, + "arg_names": ["section", "comment_id"], + }, + "prd_create_section": { + "description": "Create a new section in the project.", + "input_schema": { + "type": "object", + "properties": { + "slug": {"type": "string"}, + "title": {"type": "string"}, + "section_type": {"type": "string"}, + "content": {"type": "string"}, + "summary": {"type": "string"}, + "tags": {"type": "array", "items": {"type": "string"}}, + "notes": {"type": "string"}, + "sort_order": {"type": "integer"}, + }, + "required": ["slug", "title"], + "additionalProperties": False, + }, + "arg_names": ["slug", "title", "section_type", "content", "summary", "tags", "notes", "sort_order"], + }, + "prd_delete_section": { + "description": "Delete a section (warns about dependent sections).", + "input_schema": { + "type": "object", + "properties": { + "section": {"type": "string"}, + }, + "required": ["section"], + "additionalProperties": False, + }, + "arg_names": ["section"], + }, + "prd_add_dependency": { + "description": "Add dependency between two sections (idempotent).", + "input_schema": { + "type": "object", + "properties": { + "section": {"type": "string"}, + "depends_on": {"type": "string"}, + "dependency_type": {"type": "string"}, + "description": {"type": "string"}, + }, + "required": ["section", "depends_on"], + "additionalProperties": False, + }, + "arg_names": ["section", "depends_on", "dependency_type", "description"], + }, + "prd_remove_dependency": { + "description": "Remove dependency between two sections.", + "input_schema": { + "type": "object", + "properties": { + "section": {"type": "string"}, + "depends_on": {"type": "string"}, + }, + "required": ["section", "depends_on"], + "additionalProperties": False, + }, + "arg_names": ["section", "depends_on"], + }, + "prd_suggest_dependencies": { + "description": "Suggest dependencies for a section using content similarity.", + "input_schema": { + "type": "object", + "properties": { + "section": {"type": "string"}, + }, + "required": ["section"], + "additionalProperties": False, + }, + "arg_names": ["section"], + }, + "prd_get_changelog": { + "description": "Get recent revision history across all sections.", + "input_schema": { + "type": "object", + "properties": { + "limit": {"type": "integer"}, + }, + "additionalProperties": False, + }, + "arg_names": ["limit"], + }, + "prd_get_revisions": { + "description": "List revisions for a section.", + "input_schema": { + "type": "object", + "properties": { + "section": {"type": "string"}, + }, + "required": ["section"], + "additionalProperties": False, + }, + "arg_names": ["section"], + }, + "prd_rollback_section": { + "description": "Rollback section to a previous revision (saves backup first).", + "input_schema": { + "type": "object", + "properties": { + "section": {"type": "string"}, + "revision": {"type": "integer"}, + }, + "required": ["section", "revision"], + "additionalProperties": False, + }, + "arg_names": ["section", "revision"], + }, + "prd_move_section": { + "description": "Move section (change sort order or parent).", + "input_schema": { + "type": "object", + "properties": { + "section": {"type": "string"}, + "sort_order": {"type": "integer"}, + "parent_section": {"type": "string"}, + }, + "required": ["section"], + "additionalProperties": False, + }, + "arg_names": ["section", "sort_order", "parent_section"], + }, + "prd_duplicate_section": { + "description": "Duplicate a section with a new slug.", + "input_schema": { + "type": "object", + "properties": { + "section": {"type": "string"}, + "new_slug": {"type": "string"}, + "new_title": {"type": "string"}, + }, + "required": ["section", "new_slug"], + "additionalProperties": False, + }, + "arg_names": ["section", "new_slug", "new_title"], + }, + "prd_bulk_status": { + "description": "Bulk update status for multiple sections.", + "input_schema": { + "type": "object", + "properties": { + "sections": {"type": "array", "items": {"type": "string"}}, + "status": {"type": "string"}, + }, + "required": ["sections", "status"], + "additionalProperties": False, + }, + "arg_names": ["sections", "status"], + }, + "prd_export_markdown": { + "description": "Export entire project as markdown (large output).", + "input_schema": { + "type": "object", + "properties": {}, + "additionalProperties": False, + }, + "arg_names": [], + }, + "prd_import_markdown": { + "description": "Import markdown document into the project.", + "input_schema": { + "type": "object", + "properties": { + "markdown": {"type": "string"}, + "replace_existing": {"type": "boolean"}, + "heading_level": {"type": "integer"}, + "manual_delimiter": {"type": "string"}, + }, + "required": ["markdown"], + "additionalProperties": False, + }, + "arg_names": ["markdown", "replace_existing", "heading_level", "manual_delimiter"], + }, + "prd_token_stats": { + "description": "Get token savings statistics for the project.", + "input_schema": { + "type": "object", + "properties": {}, + "additionalProperties": False, + }, + "arg_names": [], + }, + "prd_get_settings": { + "description": "Get project settings.", + "input_schema": { + "type": "object", + "properties": {}, + "additionalProperties": False, + }, + "arg_names": [], + }, + "prd_update_settings": { + "description": "Update project settings.", + "input_schema": { + "type": "object", + "properties": { + "settings": {"type": "object"}, + }, + "required": ["settings"], + "additionalProperties": False, + }, + "arg_names": ["settings"], + }, } APPROVAL_ALLOWED_TOOLS = {f"mcp__prd-forge__{k}" for k in CHAT_ALLOWED_MCP_TOOLS} @@ -860,16 +1103,27 @@ async def _resolve_project_id_or_none(slug: str): return await pool.fetchval("SELECT id FROM projects WHERE slug = $1", slug) -async def _get_or_create_project_chat(project_id): +async def _get_or_create_project_chat(project_id, chat_type="main", section_id=None): + if section_id: + return await pool.fetchval( + """ + INSERT INTO project_chats (project_id, chat_type, section_id) + VALUES ($1, $2, $3) + ON CONFLICT (project_id, chat_type, COALESCE(section_id, '00000000-0000-0000-0000-000000000000')) + DO UPDATE SET updated_at = now() + RETURNING id + """, + project_id, chat_type, section_id, + ) return await pool.fetchval( """ - INSERT INTO project_chats (project_id) - VALUES ($1) - ON CONFLICT (project_id) + INSERT INTO project_chats (project_id, chat_type) + VALUES ($1, $2) + ON CONFLICT (project_id, chat_type, COALESCE(section_id, '00000000-0000-0000-0000-000000000000')) DO UPDATE SET updated_at = now() RETURNING id """, - project_id, + project_id, chat_type, ) @@ -1116,11 +1370,6 @@ async def _ensure_chat_generated_project_graph_data(project_id) -> None: -@app.get("/", response_class=HTMLResponse) -async def index(): - return HTML - - @app.get("/api/projects") async def list_projects(): rows = await pool.fetch(""" @@ -1134,8 +1383,19 @@ async def list_projects(): return [row_dict(r) for r in rows] +@app.get("/api/templates") +async def get_templates(): + return list_templates() + + @app.post("/api/projects") async def create_project(request: Request): + from auth import require_authenticated_user + + user = await require_authenticated_user(request, pool) + if isinstance(user, JSONResponse): + return user + try: body = await request.json() except Exception: @@ -1154,22 +1414,25 @@ async def create_project(request: Request): ) description = str(body.get("description") or "").strip() + template_id = str(body.get("template_id") or "").strip() or None try: - row = await pool.fetchrow( - """ - INSERT INTO projects (name, slug, description) - VALUES ($1, $2, $3) - RETURNING id, slug, name, description, version, created_at, updated_at - """, + result = await create_project_with_template( + pool, name, slug, description, + template_id=template_id, + user_id=user.get("user_id"), ) + return result + except ValueError as e: + return JSONResponse({"error": str(e)}, 400) except asyncpg.UniqueViolationError: return JSONResponse({"error": f"project slug '{slug}' already exists"}, 409) - - return row_dict(row) + except Exception as e: + logger.error("create_project: %s", e) + return JSONResponse({"error": "internal error"}, 500) @app.get("/api/projects/{slug}") @@ -1307,6 +1570,27 @@ async def patch_section(slug: str, section: str, request: Request): return JSONResponse({"error": f"section '{section}' not found"}, 404) body = await request.json() + + # Optimistic locking: if client sends expected_revision, verify it matches + expected_rev = body.get("expected_revision") + if expected_rev is not None: + current_rev = await pool.fetchval( + "SELECT COALESCE(MAX(revision_number), 0) FROM section_revisions WHERE section_id = $1", + sec["id"], + ) + if current_rev != expected_rev: + return JSONResponse({ + "error": { + "code": "CONFLICT", + "message": f"Section was updated (revision {expected_rev} → {current_rev}). Reload and retry.", + "status": 409, + "details": { + "current_revision": current_rev, + "expected_revision": expected_rev, + }, + } + }, 409) + allowed = {"status", "tags", "title", "summary"} updates = {k: v for k, v in body.items() if k in allowed and v is not None} if not updates: @@ -1327,11 +1611,19 @@ async def patch_section(slug: str, section: str, request: Request): await pool.execute( f"UPDATE sections SET {', '.join(set_parts)} WHERE id = $1", *params ) + await broadcast_project_event(slug, "section_updated", {"section": section, "fields": list(updates.keys())}) return {"ok": True, "updated": list(updates.keys())} @app.post("/api/projects/{slug}/sections/{section}/notes") async def update_notes(slug: str, section: str, request: Request): + from auth import require_project_access + + access = await require_project_access(request, pool, slug, min_role="editor") + user, role = access + if isinstance(user, JSONResponse): + return user + body = await request.json() notes = body.get("notes", "") proj = await pool.fetchrow("SELECT id FROM projects WHERE slug = $1", slug) @@ -1362,6 +1654,7 @@ async def create_comment(slug: str, section: str, request: Request): VALUES ($1, $2, $3, $4, $5) RETURNING * """, sec_id, body["anchor_text"], body.get("anchor_prefix", ""), body.get("anchor_suffix", ""), body["body"]) + await broadcast_project_event(slug, "comment_added", {"section": section, "comment_id": str(row["id"])}) return row_dict(row) @@ -1499,56 +1792,144 @@ async def get_token_stats(slug: str): await _backfill_missing_initial_revisions(pid) - totals = await pool.fetchrow(""" - SELECT COUNT(*) AS operations, - COALESCE(SUM(full_doc_tokens), 0) AS total_full, - COALESCE(SUM(loaded_tokens), 0) AS total_loaded - FROM token_estimates WHERE project_id = $1 + # Full document size (current) + full_doc = await pool.fetchrow(""" + SELECT COALESCE(SUM(word_count), 0) AS total_words, COUNT(*) AS section_count + FROM sections WHERE project_id = $1 """, pid) + full_doc_words = full_doc["total_words"] + full_doc_tokens = int(full_doc_words * 1.3) + # Check if new access log has data + has_access_log = await pool.fetchval( + "SELECT EXISTS(SELECT 1 FROM section_access_log WHERE project_id = $1)", pid + ) + + if has_access_log: + # --- Honest session-based calculation --- + sessions = await pool.fetch(""" + WITH numbered AS ( + SELECT *, + CASE WHEN created_at - LAG(created_at) OVER (ORDER BY created_at) + > interval '30 minutes' + OR LAG(created_at) OVER (ORDER BY created_at) IS NULL + THEN 1 ELSE 0 END AS new_session + FROM section_access_log WHERE project_id = $1 + ), + sessioned AS ( + SELECT *, SUM(new_session) OVER (ORDER BY created_at) AS session_id + FROM numbered + ), + session_coverage AS ( + SELECT session_id, section_id, + MAX(CASE access_level + WHEN 'full' THEN 1.0 WHEN 'summary' THEN 0.10 WHEN 'snippet' THEN 0.15 + END) AS coverage + FROM sessioned GROUP BY session_id, section_id + ), + session_stats AS ( + SELECT sc.session_id, + $2::int AS full_doc_words, + SUM(COALESCE(s.word_count, 0) * sc.coverage)::int AS unique_loaded_words, + COUNT(DISTINCT sc.section_id) AS sections_touched + FROM session_coverage sc + LEFT JOIN sections s ON s.id = sc.section_id + GROUP BY sc.session_id + ) + SELECT + session_id, + full_doc_words, + unique_loaded_words, + sections_touched, + CASE WHEN full_doc_words > 0 + THEN ROUND((1.0 - unique_loaded_words::numeric / full_doc_words) * 100, 1) + ELSE 0 END AS savings_pct + FROM session_stats ORDER BY session_id + """, pid, full_doc_words) + + total_ops = await pool.fetchval( + "SELECT COUNT(*) FROM section_access_log WHERE project_id = $1", pid + ) + session_count = len(sessions) + avg_savings = round(sum(s["savings_pct"] for s in sessions) / session_count, 1) if session_count > 0 else 0 + best_session = max((s["savings_pct"] for s in sessions), default=0) + avg_sections_per_session = round(sum(s["sections_touched"] for s in sessions) / session_count, 1) if session_count > 0 else 0 + total_unique_loaded = sum(s["unique_loaded_words"] for s in sessions) + total_loaded_tokens = int(total_unique_loaded * 1.3) + total_saved_tokens = max(0, full_doc_tokens * session_count - total_loaded_tokens) + + # Section heatmap — how often each section is accessed + heatmap = await pool.fetch(""" + SELECT s.slug, s.title, COUNT(*) AS access_count, + MAX(CASE access_level WHEN 'full' THEN 1 WHEN 'summary' THEN 0 WHEN 'snippet' THEN 0 END) AS has_full_read + FROM section_access_log sal + JOIN sections s ON s.id = sal.section_id + WHERE sal.project_id = $1 + GROUP BY s.slug, s.title + ORDER BY access_count DESC + """, pid) + else: + # --- Fallback to legacy token_estimates --- + totals = await pool.fetchrow(""" + SELECT COUNT(*) AS operations, + COALESCE(SUM(full_doc_tokens), 0) AS total_full, + COALESCE(SUM(loaded_tokens), 0) AS total_loaded + FROM token_estimates WHERE project_id = $1 + """, pid) + total_full_legacy = totals["total_full"] + total_loaded_legacy = totals["total_loaded"] + saved_legacy = total_full_legacy - total_loaded_legacy + avg_savings = round(saved_legacy / total_full_legacy * 100, 1) if total_full_legacy > 0 else 0 + total_ops = totals["operations"] + session_count = 0 + best_session = avg_savings + avg_sections_per_session = 0 + total_loaded_tokens = total_loaded_legacy + total_saved_tokens = saved_legacy + heatmap = [] + + # By operation (from legacy table — still populated) by_op = await pool.fetch(""" - SELECT operation, - COUNT(*) AS count, + SELECT operation, COUNT(*) AS count, SUM(full_doc_tokens) AS full_tokens, SUM(loaded_tokens) AS loaded_tokens FROM token_estimates WHERE project_id = $1 GROUP BY operation ORDER BY count DESC """, pid) + # Daily trend daily = await pool.fetch(""" SELECT d.day::date AS day, COALESCE(COUNT(t.id), 0) AS operations, COALESCE(SUM(t.full_doc_tokens - t.loaded_tokens), 0) AS tokens_saved FROM generate_series(current_date - 6, current_date, '1 day') AS d(day) - LEFT JOIN ( - SELECT * FROM token_estimates WHERE project_id = $1 - ) t ON t.created_at::date = d.day + LEFT JOIN (SELECT * FROM token_estimates WHERE project_id = $1) t + ON t.created_at::date = d.day GROUP BY d.day ORDER BY d.day ASC """, pid) - total_full = totals["total_full"] - total_loaded = totals["total_loaded"] - saved = total_full - total_loaded - pct = round(saved / total_full * 100, 1) if total_full > 0 else 0 - - project_stats = await pool.fetchrow( - """ + project_stats = await pool.fetchrow(""" SELECT (SELECT COUNT(*) FROM sections WHERE project_id = $1) AS section_count, (SELECT COUNT(*) FROM section_dependencies WHERE project_id = $1) AS dependency_count, (SELECT COUNT(*) FROM section_revisions r - JOIN sections s ON s.id = r.section_id - WHERE s.project_id = $1) AS revision_count - """, - pid, - ) + JOIN sections s ON s.id = r.section_id WHERE s.project_id = $1) AS revision_count + """, pid) + + activity_rows = await pool.fetch(""" + SELECT tool_name, detail, created_at FROM mcp_activity + WHERE project_id = $1 ORDER BY created_at DESC LIMIT 50 + """, pid) return { - "operations": totals["operations"], - "total_full_doc_tokens": total_full, - "total_loaded_tokens": total_loaded, - "total_saved_tokens": saved, - "savings_percent": pct, + "operations": total_ops, + "total_full_doc_tokens": full_doc_tokens, + "total_loaded_tokens": total_loaded_tokens, + "total_saved_tokens": total_saved_tokens, + "savings_percent": float(avg_savings), + "sessions": session_count, + "best_session_savings": float(best_session), + "avg_sections_per_session": float(avg_sections_per_session), "by_operation": [row_dict(r) for r in by_op], "daily_trend": [{"day": str(r["day"]), "operations": r["operations"], "tokens_saved": r["tokens_saved"]} for r in daily], @@ -1557,6 +1938,8 @@ async def get_token_stats(slug: str): "dependencies": project_stats["dependency_count"], "revisions": project_stats["revision_count"], }, + "activity": [row_dict(r) for r in activity_rows], + "section_heatmap": [row_dict(r) for r in heatmap] if heatmap else [], } @@ -1583,21 +1966,6 @@ async def chat_provider_status(): } -@app.put("/api/chat/api-key") -async def set_chat_api_key(request: Request): - """Set Anthropic API key at runtime (not persisted to disk).""" - global _runtime_anthropic_api_key - try: - body = await request.json() - except Exception: - return JSONResponse({"error": "invalid JSON body"}, 400) - key = str(body.get("api_key") or "").strip() - if key and not key.startswith("sk-ant-"): - return JSONResponse({"error": "Invalid API key format (expected sk-ant-...)"}, 400) - _runtime_anthropic_api_key = key - return {"ok": True, "configured": bool(key), "key_hint": f"...{key[-4:]}" if len(key) >= 4 else ""} - - @app.post("/api/chat/cli-login") async def cli_login(): """Generate PKCE OAuth URL for Claude CLI authentication.""" @@ -2041,6 +2409,259 @@ async def export_project(slug: str): return PlainTextResponse("\n".join(lines), media_type="text/plain") +# --- Member management --- + +@app.get("/api/projects/{slug}/members") +async def list_project_members(slug: str, request: Request): + """List all members of a project.""" + from auth import require_project_access + + access = await require_project_access(request, pool, slug, min_role="viewer") + user, role = access + if isinstance(user, JSONResponse): + return user + + proj = await pool.fetchrow("SELECT id FROM projects WHERE slug = $1", slug) + if not proj: + return JSONResponse({"error": f"project '{slug}' not found"}, 404) + rows = await pool.fetch(""" + SELECT pm.id, pm.user_id, pm.role, pm.created_at, pm.updated_at + FROM project_members pm + WHERE pm.project_id = $1 + ORDER BY pm.created_at + """, proj["id"]) + return [row_dict(r) for r in rows] + + +@app.post("/api/projects/{slug}/members") +async def add_project_member(slug: str, request: Request): + """Add a member to a project.""" + from auth import require_project_access + + access = await require_project_access(request, pool, slug, min_role="admin") + user, role = access + if isinstance(user, JSONResponse): + return user + + proj = await pool.fetchrow("SELECT id FROM projects WHERE slug = $1", slug) + if not proj: + return JSONResponse({"error": f"project '{slug}' not found"}, 404) + try: + body = await request.json() + except Exception: + return JSONResponse({"error": "invalid JSON body"}, 400) + user_id = body.get("user_id") + role = body.get("role", "viewer") + if not user_id: + return JSONResponse({"error": "user_id is required"}, 400) + valid_roles = {"owner", "admin", "editor", "commenter", "viewer"} + if role not in valid_roles: + return JSONResponse({"error": f"invalid role, must be one of {sorted(valid_roles)}"}, 400) + try: + row = await pool.fetchrow(""" + INSERT INTO project_members (project_id, user_id, role) + VALUES ($1, $2, $3) + ON CONFLICT (project_id, user_id) DO UPDATE SET role = EXCLUDED.role + RETURNING id, user_id, role, created_at + """, proj["id"], _uuid.UUID(user_id), role) + return row_dict(row) + except Exception: + return JSONResponse({"error": "internal error"}, 500) + + +@app.delete("/api/projects/{slug}/members/{user_id}") +async def remove_project_member(slug: str, user_id: str, request: Request): + """Remove a member from a project.""" + from auth import require_project_access + + access = await require_project_access(request, pool, slug, min_role="admin") + user, role = access + if isinstance(user, JSONResponse): + return user + + proj = await pool.fetchrow("SELECT id FROM projects WHERE slug = $1", slug) + if not proj: + return JSONResponse({"error": f"project '{slug}' not found"}, 404) + result = await pool.execute( + "DELETE FROM project_members WHERE project_id = $1 AND user_id = $2", + proj["id"], _uuid.UUID(user_id), + ) + removed = result.split()[-1] != "0" + return {"removed": removed} + + +# --- Audit events --- + +@app.get("/api/projects/{slug}/audit") +async def list_audit_events(slug: str, limit: int = 50): + """List recent audit events for a project.""" + proj = await pool.fetchrow("SELECT id FROM projects WHERE slug = $1", slug) + if not proj: + return JSONResponse({"error": f"project '{slug}' not found"}, 404) + rows = await pool.fetch(""" + SELECT id, user_id, action, resource, detail, created_at + FROM audit_events + WHERE project_id = $1 + ORDER BY created_at DESC + LIMIT $2 + """, proj["id"], limit) + return [row_dict(r) for r in rows] + + +# --- WebSocket token --- + +@app.post("/api/ws-token") +async def create_ws_token(request: Request): + """Mint a short-lived HMAC token for WebSocket authentication.""" + from auth import require_project_access + from ws import mint_ws_token + + try: + body = await request.json() + except Exception: + return JSONResponse({"error": "invalid JSON body"}, 400) + + project_slug = body.get("project_slug") + if not project_slug: + return JSONResponse({"error": "project_slug required"}, 400) + + # Verify project exists + proj = await pool.fetchrow("SELECT id FROM projects WHERE slug = $1", project_slug) + if not proj: + return JSONResponse({"error": f"project '{project_slug}' not found"}, 404) + + access = await require_project_access(request, pool, project_slug, min_role="viewer") + user, role = access + if isinstance(user, JSONResponse): + return user + + # user_id is None in pre-setup mode (anonymous), str otherwise + user_id = user.get("user_id") or "anonymous" + token = mint_ws_token(str(user_id), project_slug) + return {"token": token} + + +# --- WebSocket handler --- + +# Connected clients: {project_slug: {user_id: WebSocket}} +_ws_connections: dict[str, dict[str, WebSocket]] = {} +_ws_redis_warned = [False] # mutable holder — warn once, no global needed + + +@app.websocket("/ws/projects/{slug}") +async def websocket_project(websocket: WebSocket, slug: str): + """WebSocket endpoint for real-time project updates and presence.""" + from auth import _is_auth_enforced, get_user_project_role + from ws import verify_ws_token + + token = websocket.query_params.get("token") + if not token: + await websocket.close(code=4001, reason="Missing token") + return + + payload = verify_ws_token(token) + if not payload: + await websocket.close(code=4001, reason="Invalid or expired token") + return + + if payload.get("project") != slug: + await websocket.close(code=4003, reason="Token project mismatch") + return + + user_id = payload["sub"] + jti = payload.get("jti", "") + + # jti uniqueness: Redis SET NX EX — reject replayed tokens + if redis_client and jti: + was_set = await redis_client.set(f"ws:jti:{jti}", "1", nx=True, ex=120) + if not was_set: + await websocket.close(code=4002, reason="Token already used") + return + elif not redis_client and jti: + if not _ws_redis_warned[0]: + logger.warning("Redis unavailable — WS token replay protection disabled") + _ws_redis_warned[0] = True + + # Membership re-check: user may have been removed since token was minted + if await _is_auth_enforced(pool): + ws_role = await get_user_project_role(pool, user_id, slug) + if not ws_role: + await websocket.close(code=4003, reason="No project access") + return + + await websocket.accept() + + # Register connection + if slug not in _ws_connections: + _ws_connections[slug] = {} + _ws_connections[slug][user_id] = websocket + + # Broadcast presence update + await _broadcast_presence(slug) + + try: + while True: + data = await websocket.receive_text() + # Handle incoming messages (e.g., cursor position, typing indicator) + try: + msg = json.loads(data) + if msg.get("type") == "presence_update": + # Update user's active section and re-broadcast + await _broadcast_presence(slug) + except json.JSONDecodeError: + pass + except WebSocketDisconnect: + pass + finally: + # Unregister + if slug in _ws_connections: + _ws_connections[slug].pop(user_id, None) + if not _ws_connections[slug]: + del _ws_connections[slug] + else: + await _broadcast_presence(slug) + + +async def _broadcast_presence(slug: str): + """Send presence list to all connected clients for a project.""" + conns = _ws_connections.get(slug, {}) + users = [{"id": uid, "name": uid} for uid in conns] + message = json.dumps({"type": "presence_update", "data": {"users": users}}) + disconnected = [] + for uid, ws in conns.items(): + try: + await ws.send_text(message) + except Exception: + disconnected.append(uid) + for uid in disconnected: + conns.pop(uid, None) + + +async def broadcast_project_event(slug: str, event_type: str, data: dict): + """Broadcast a real-time event to all connected project clients. + + Sends via local WS connections AND Redis pub/sub for multi-process. + """ + message = json.dumps({"type": event_type, "data": data}) + # Local broadcast + conns = _ws_connections.get(slug, {}) + if conns: + disconnected = [] + for uid, ws in conns.items(): + try: + await ws.send_text(message) + except Exception: + disconnected.append(uid) + for uid in disconnected: + conns.pop(uid, None) + # Redis pub/sub for multi-process + if redis_client: + try: + await redis_client.publish(f"project:{slug}", message) + except Exception as e: + logger.warning("Redis publish failed: %s", e) + + @app.get("/health") async def health(): try: diff --git a/api/auth.py b/api/auth.py new file mode 100644 index 0000000..f5bb050 --- /dev/null +++ b/api/auth.py @@ -0,0 +1,181 @@ +"""Python-side auth: read-only consumer of Better Auth tables. + +All auth write operations happen in Next.js. Python API validates +sessions and resolves roles by querying the auth tables directly. +""" + +import logging +from datetime import datetime, timezone +from functools import wraps + +from fastapi import Request +from fastapi.responses import JSONResponse + +logger = logging.getLogger("prd_forge_auth") + +# Actual Better Auth table/column names (verified by contract test) +AUTH_TABLES = { + "user": "user", + "session": "session", + "account": "account", + "organization": "organization", + "member": "member", + "invitation": "invitation", + "verification": "verification", +} + + +async def get_session_user(request: Request, pool): + """Extract user from session token in cookie or Authorization header. + + Returns dict with user info or None if not authenticated. + """ + # Try cookie first (browser), then Authorization header (API) + token = None + cookie = request.cookies.get("better-auth.session_token") + if cookie: + # Cookie value may have .signature suffix + token = cookie.split(".")[0] if "." in cookie else cookie + else: + auth_header = request.headers.get("authorization", "") + if auth_header.startswith("Bearer "): + token = auth_header[7:] + + if not token: + return None + + row = await pool.fetchrow( + f""" + SELECT s.id AS session_id, s."userId" AS user_id, s."expiresAt", + u.name, u.email, u.image + FROM "{AUTH_TABLES['session']}" s + JOIN "{AUTH_TABLES['user']}" u ON u.id = s."userId" + WHERE s.token = $1 + """, + token, + ) + + if not row: + return None + + # Check expiry + expires = row["expiresAt"] + if expires and expires.replace(tzinfo=timezone.utc) < datetime.now(timezone.utc): + return None + + return { + "session_id": row["session_id"], + "user_id": row["user_id"], + "name": row["name"], + "email": row["email"], + "image": row["image"], + } + + +async def get_user_project_role(pool, user_id: str, project_slug: str) -> str | None: + """Get user's role for a project. + + Checks project_members first, then falls back to org membership. + Returns role string or None if no access. + """ + # Direct project membership + role = await pool.fetchval( + """ + SELECT pm.role FROM project_members pm + JOIN projects p ON p.id = pm.project_id + WHERE pm.user_id = $1 AND p.slug = $2 + """, + user_id, + project_slug, + ) + if role: + return role + + # Org membership fallback: org owner/admin → project admin + org_role = await pool.fetchval( + f""" + SELECT m.role FROM "{AUTH_TABLES['member']}" m + JOIN "{AUTH_TABLES['organization']}" o ON o.id = m."organizationId" + JOIN projects p ON p.organization_id = o.id::uuid + WHERE m."userId" = $1 AND p.slug = $2 + """, + user_id, + project_slug, + ) + if org_role in ("owner", "admin"): + return "admin" + if org_role == "member": + return "editor" + + return None + + +# Role hierarchy for permission checks +ROLE_HIERARCHY = { + "owner": 5, + "admin": 4, + "editor": 3, + "commenter": 2, + "viewer": 1, +} + + +def has_min_role(user_role: str, min_role: str) -> bool: + """Check if user_role meets the minimum required role.""" + return ROLE_HIERARCHY.get(user_role, 0) >= ROLE_HIERARCHY.get(min_role, 0) + + +async def require_authenticated_user(request: Request, pool): + """Check user is authenticated. Returns user dict or JSONResponse error. + + Does NOT check project role — use for endpoints where the project doesn't exist yet. + During migration / pre-setup: if auth not bootstrapped, returns a local-mode user. + """ + from errors import unauthorized + + if not await _is_auth_enforced(pool): + return {"user_id": None, "name": "local", "email": ""} + + user = await get_session_user(request, pool) + if not user: + return unauthorized() + + return user + + +async def _is_auth_enforced(pool) -> bool: + """Return True if auth should be enforced (tables exist AND setup completed).""" + tbl = AUTH_TABLES["session"] + auth_exists = await pool.fetchval(f"""SELECT to_regclass('public."{tbl}"')""") + if not auth_exists: + return False + bootstrap_exists = await pool.fetchval("SELECT to_regclass('public.prdforge_bootstrap')") + if bootstrap_exists: + has_bootstrap = await pool.fetchval("SELECT 1 FROM prdforge_bootstrap LIMIT 1") + if not has_bootstrap: + return False + return True + + +async def require_project_access(request: Request, pool, slug: str, min_role: str = "viewer"): + """Check user has required role for project. Returns (user, role) or JSONResponse error. + + During migration / pre-setup: allow all access (single-user mode). + """ + from errors import unauthorized, permission_denied, not_found + + if not await _is_auth_enforced(pool): + return {"user_id": None, "name": "local", "email": ""}, "owner" + + user = await get_session_user(request, pool) + if not user: + return unauthorized(), None + + role = await get_user_project_role(pool, user["user_id"], slug) + if not role: + return not_found("project", slug), None + + if not has_min_role(role, min_role): + return permission_denied(f"Requires {min_role} role, you have {role}"), None + + return user, role diff --git a/api/auth_contract.py b/api/auth_contract.py new file mode 100644 index 0000000..0ac531a --- /dev/null +++ b/api/auth_contract.py @@ -0,0 +1,50 @@ +"""Contract test: verify Better Auth table names and columns exist. + +Better Auth uses Prisma model names which may differ from DB table names. +This contract verifies the ACTUAL tables created by `prisma migrate deploy`. +Pin: better-auth@1.4.7 +""" + +# Expected tables and their required columns (from Prisma schema @@map) +EXPECTED_TABLES = { + "user": ["id", "name", "email", "emailVerified", "createdAt", "updatedAt"], + "session": ["id", "token", "userId", "expiresAt", "createdAt"], + "account": ["id", "accountId", "providerId", "userId", "createdAt"], + "verification": ["id", "identifier", "value", "expiresAt"], + "organization": ["id", "name", "slug", "createdAt", "updatedAt"], + "member": ["id", "organizationId", "userId", "role", "createdAt"], + "invitation": ["id", "organizationId", "email", "role", "status", "expiresAt"], +} + + +async def verify_auth_contract(pool) -> list[str]: + """Verify Better Auth tables exist with expected columns. + + Returns list of errors (empty = all good). + """ + errors = [] + + for table, expected_cols in EXPECTED_TABLES.items(): + # Check table exists + exists = await pool.fetchval( + "SELECT to_regclass($1)", table + ) + if not exists: + errors.append(f"Table '{table}' does not exist") + continue + + # Check columns + actual_cols = await pool.fetch( + """ + SELECT column_name FROM information_schema.columns + WHERE table_name = $1 AND table_schema = 'public' + """, + table, + ) + actual_col_names = {r["column_name"] for r in actual_cols} + + for col in expected_cols: + if col not in actual_col_names: + errors.append(f"Table '{table}' missing column '{col}'") + + return errors diff --git a/api/errors.py b/api/errors.py new file mode 100644 index 0000000..091e928 --- /dev/null +++ b/api/errors.py @@ -0,0 +1,68 @@ +"""Structured error responses for the Python API. + +Format: {"error": {"code": "...", "message": "...", "status": N, "details": {}}} + +9 standard error codes matching the PRD spec. +""" + +from fastapi.responses import JSONResponse + + +# Standard error codes +UNAUTHORIZED = "UNAUTHORIZED" +PERMISSION_DENIED = "PERMISSION_DENIED" +NOT_FOUND = "NOT_FOUND" +VALIDATION_ERROR = "VALIDATION_ERROR" +CONFLICT = "CONFLICT" +RATE_LIMITED = "RATE_LIMITED" +INTERNAL_ERROR = "INTERNAL_ERROR" +CHAT_DISABLED = "CHAT_DISABLED" +NO_API_KEY = "NO_API_KEY" + + +def error_response( + code: str, + message: str, + status: int, + details: dict | None = None, +) -> JSONResponse: + """Create a structured error response.""" + return JSONResponse( + { + "error": { + "code": code, + "message": message, + "status": status, + "details": details or {}, + } + }, + status_code=status, + ) + + +def not_found(resource: str, identifier: str) -> JSONResponse: + return error_response( + NOT_FOUND, + f"{resource} '{identifier}' not found", + 404, + ) + + +def validation_error(message: str, details: dict | None = None) -> JSONResponse: + return error_response(VALIDATION_ERROR, message, 400, details) + + +def unauthorized(message: str = "Authentication required") -> JSONResponse: + return error_response(UNAUTHORIZED, message, 401) + + +def permission_denied(message: str = "Insufficient permissions") -> JSONResponse: + return error_response(PERMISSION_DENIED, message, 403) + + +def conflict(message: str, details: dict | None = None) -> JSONResponse: + return error_response(CONFLICT, message, 409, details) + + +def internal_error(message: str = "Internal server error") -> JSONResponse: + return error_response(INTERNAL_ERROR, message, 500) diff --git a/ui/requirements.txt b/api/requirements.txt similarity index 65% rename from ui/requirements.txt rename to api/requirements.txt index 85650cf..c3cce95 100644 --- a/ui/requirements.txt +++ b/api/requirements.txt @@ -2,4 +2,6 @@ fastapi>=0.115 uvicorn>=0.34 asyncpg>=0.29 httpx>=0.28 +redis[hiredis]>=5.0 +websockets>=14.0 mcp[cli]>=1.9 diff --git a/ui/static/MARKED_VERSION b/api/static/MARKED_VERSION similarity index 100% rename from ui/static/MARKED_VERSION rename to api/static/MARKED_VERSION diff --git a/ui/static/fonts.css b/api/static/fonts.css similarity index 100% rename from ui/static/fonts.css rename to api/static/fonts.css diff --git a/ui/static/fonts/inter-cyrillic-ext.woff2 b/api/static/fonts/inter-cyrillic-ext.woff2 similarity index 100% rename from ui/static/fonts/inter-cyrillic-ext.woff2 rename to api/static/fonts/inter-cyrillic-ext.woff2 diff --git a/ui/static/fonts/inter-cyrillic.woff2 b/api/static/fonts/inter-cyrillic.woff2 similarity index 100% rename from ui/static/fonts/inter-cyrillic.woff2 rename to api/static/fonts/inter-cyrillic.woff2 diff --git a/ui/static/fonts/inter-greek-ext.woff2 b/api/static/fonts/inter-greek-ext.woff2 similarity index 100% rename from ui/static/fonts/inter-greek-ext.woff2 rename to api/static/fonts/inter-greek-ext.woff2 diff --git a/ui/static/fonts/inter-greek.woff2 b/api/static/fonts/inter-greek.woff2 similarity index 100% rename from ui/static/fonts/inter-greek.woff2 rename to api/static/fonts/inter-greek.woff2 diff --git a/ui/static/fonts/inter-latin-ext.woff2 b/api/static/fonts/inter-latin-ext.woff2 similarity index 100% rename from ui/static/fonts/inter-latin-ext.woff2 rename to api/static/fonts/inter-latin-ext.woff2 diff --git a/ui/static/fonts/inter-latin.woff2 b/api/static/fonts/inter-latin.woff2 similarity index 100% rename from ui/static/fonts/inter-latin.woff2 rename to api/static/fonts/inter-latin.woff2 diff --git a/ui/static/fonts/inter-vietnamese.woff2 b/api/static/fonts/inter-vietnamese.woff2 similarity index 100% rename from ui/static/fonts/inter-vietnamese.woff2 rename to api/static/fonts/inter-vietnamese.woff2 diff --git a/ui/static/fonts/jetbrains-mono-cyrillic-ext.woff2 b/api/static/fonts/jetbrains-mono-cyrillic-ext.woff2 similarity index 100% rename from ui/static/fonts/jetbrains-mono-cyrillic-ext.woff2 rename to api/static/fonts/jetbrains-mono-cyrillic-ext.woff2 diff --git a/ui/static/fonts/jetbrains-mono-cyrillic.woff2 b/api/static/fonts/jetbrains-mono-cyrillic.woff2 similarity index 100% rename from ui/static/fonts/jetbrains-mono-cyrillic.woff2 rename to api/static/fonts/jetbrains-mono-cyrillic.woff2 diff --git a/ui/static/fonts/jetbrains-mono-greek.woff2 b/api/static/fonts/jetbrains-mono-greek.woff2 similarity index 100% rename from ui/static/fonts/jetbrains-mono-greek.woff2 rename to api/static/fonts/jetbrains-mono-greek.woff2 diff --git a/ui/static/fonts/jetbrains-mono-latin-ext.woff2 b/api/static/fonts/jetbrains-mono-latin-ext.woff2 similarity index 100% rename from ui/static/fonts/jetbrains-mono-latin-ext.woff2 rename to api/static/fonts/jetbrains-mono-latin-ext.woff2 diff --git a/ui/static/fonts/jetbrains-mono-latin.woff2 b/api/static/fonts/jetbrains-mono-latin.woff2 similarity index 100% rename from ui/static/fonts/jetbrains-mono-latin.woff2 rename to api/static/fonts/jetbrains-mono-latin.woff2 diff --git a/ui/static/fonts/jetbrains-mono-vietnamese.woff2 b/api/static/fonts/jetbrains-mono-vietnamese.woff2 similarity index 100% rename from ui/static/fonts/jetbrains-mono-vietnamese.woff2 rename to api/static/fonts/jetbrains-mono-vietnamese.woff2 diff --git a/ui/static/github-dark.min.css b/api/static/github-dark.min.css similarity index 100% rename from ui/static/github-dark.min.css rename to api/static/github-dark.min.css diff --git a/ui/static/highlight.min.js b/api/static/highlight.min.js similarity index 100% rename from ui/static/highlight.min.js rename to api/static/highlight.min.js diff --git a/ui/static/index.html b/api/static/index.html similarity index 99% rename from ui/static/index.html rename to api/static/index.html index ea6d5e6..f59b21e 100644 --- a/ui/static/index.html +++ b/api/static/index.html @@ -487,7 +487,7 @@

Project Chat

-