Added new ADRs for ChatKit framework integration and context injection patterns to provide a comprehensive resource for future development.

Author: mjunaidca

.claude/agents/engineering/chatkit-integration-agent.md

@@ -180,3 +180,4 @@ Before finalizing ChatKit integration:
 - **Skill**: `.claude/skills/engineering/chatkit-integration/SKILL.md`
 - **Implementation**: `rag-agent/chatkit_server.py`, `robolearn-interface/src/components/ChatKitWidget/`
 
+

.claude/skills/engineering/chatkit-integration/SKILL.md

@@ -425,3 +425,4 @@ const handleAskSelectedText = useCallback(async () => {
 - **ChatKit UI Spec**: `specs/008-chatkit-ui-widget/spec.md`
 - **Implementation**: `rag-agent/chatkit_server.py`, `robolearn-interface/src/components/ChatKitWidget/`
 
+

auth-server/scripts/seed-public-client.sql

@@ -32,3 +32,4 @@ ON CONFLICT (client_id) DO UPDATE SET
   metadata = EXCLUDED.metadata,
   updated_at = NOW();
 
+

auth-server/src/middleware.ts

@@ -57,3 +57,4 @@ export const config = {
   matcher: '/api/:path*',
 };
 
+

history/adr/0005-chatkit-framework-integration-architecture.md

@@ -0,0 +1,71 @@
+# ADR-0005: ChatKit Framework Integration Architecture
+
+> **Scope**: Integration architecture for OpenAI ChatKit framework with custom backend and AI agents. Includes framework extension pattern, persistence strategy, agent integration approach, and streaming response handling. These decisions affect how conversational AI is implemented across the platform.
+
+- **Status:** Accepted
+- **Date:** 2025-01-27
+- **Feature:** 007-chatkit-server, 008-chatkit-ui-widget
+- **Context:** RoboLearn platform needs conversational AI interface for educational assistance. Two architectural options emerged: (1) build custom chat system from scratch, or (2) integrate OpenAI ChatKit framework. ChatKit chosen to leverage standardized protocol, pre-built React components, and proven conversation management. System must integrate with existing RAG search agent, maintain conversation history, handle user/page context, and scale horizontally. Framework requires custom integration for agent execution, context injection, and authentication.
+
+<!-- Significance checklist (ALL must be true to justify this ADR)
+     1) Impact: ✅ Defines conversational AI architecture for entire platform, affects all future chat features
+     2) Alternatives: ✅ Custom implementation vs ChatKit framework, PostgreSQL vs Redis/Memory, streaming vs batch
+     3) Scope: ✅ Cross-cutting decision affecting backend, frontend, database, and agent integration
+-->
+
+## Decision
+
+Integrate **OpenAI ChatKit Framework** with the following architecture cluster:
+
+- **Framework**: OpenAI ChatKit SDK (`chatkit` package) for protocol handling and React components
+- **Backend Pattern**: Extend `ChatKitServer[RequestContext]` class, override `respond()` method for custom agent execution
+- **Persistence**: PostgreSQL 14+ via async SQLAlchemy with connection pooling
+- **Agent Integration**: OpenAI Agents SDK with custom tools (RAG search), context injection in system prompt
+- **Response Handling**: Server-Sent Events (SSE) streaming for real-time UX
+- **Context Transmission**: User info and page context via request metadata, extracted in backend and included in agent prompt
+
+## Consequences
+
+### Positive
+
+- **Standardized Protocol**: ChatKit provides proven conversation management protocol, reducing custom protocol development
+- **Pre-built UI Components**: React components reduce frontend development time, ensure consistent UX
+- **Attachment Handling**: Built-in support for file uploads/downloads without custom implementation
+- **Extensibility**: Can integrate any agent framework via `respond()` method override
+- **Scalability**: PostgreSQL handles concurrent conversations, connection pooling optimizes performance
+- **Real-time UX**: Streaming responses provide immediate feedback, better perceived performance
+- **Framework Agnostic**: Backend pattern works with any agent framework (OpenAI Agents SDK, LangChain, custom)
+
+### Negative
+
+- **Dependency on ChatKit SDK**: Vendor lock-in to OpenAI's ChatKit framework, protocol constraints
+- **Database Dependency**: Requires PostgreSQL setup and maintenance, connection management complexity
+- **Prompt Length Growth**: History included as string in prompt, grows with conversation length (mitigated by limit=10)
+- **Streaming Complexity**: More complex error handling and connection management than batch responses
+- **Learning Curve**: Team must learn ChatKit protocol and extension patterns
+- **Framework Updates**: ChatKit SDK updates may require code changes
+
+## Alternatives Considered
+
+**Alternative Stack A: Custom Chat Implementation**
+- Build custom chat protocol, React components, and persistence layer from scratch
+- **Why rejected**: Significant development time, no standardized UX, must handle all edge cases (attachments, threading, persistence)
+
+**Alternative Stack B: ChatKit + Redis/Memory Storage**
+- Use ChatKit framework but store conversations in Redis or in-memory
+- **Why rejected**: Conversations lost on restart (Redis) or server restart (memory), no persistence for user experience
+
+**Alternative Stack C: ChatKit + Batch Responses**
+- Use ChatKit but return complete responses instead of streaming
+- **Why rejected**: Poor UX (users wait for complete response), slower perceived performance
+
+**Alternative Stack D: LangChain Chat Integration**
+- Use LangChain's chat integration instead of ChatKit
+- **Why rejected**: No standardized React components, more complex setup, less proven for production use
+
+## References
+
+- Feature Spec: `specs/007-chatkit-server/spec.md` (ADR-001, ADR-002, ADR-004)
+- Implementation Plan: `specs/007-chatkit-server/plan.md` (Architecture Overview, Technology Stack)
+- Related ADRs: ADR-0006 (Context Injection Pattern)
+- Evaluator Evidence: `history/prompts/007-chatkit-server/0001-chatkit-server-reverse-engineering.spec.prompt.md`

history/adr/0006-context-injection-pattern-for-conversational-ai.md

@@ -0,0 +1,73 @@
+# ADR-0006: Context Injection Pattern for Conversational AI
+
+> **Scope**: Pattern for injecting user profile, page context, and conversation history into conversational AI agent prompts. Includes client-side context extraction, request metadata transmission, backend extraction, and prompt construction. This pattern affects how agents understand user context and provide personalized responses.
+
+- **Status:** Accepted
+- **Date:** 2025-01-27
+- **Feature:** 007-chatkit-server, 008-chatkit-ui-widget
+- **Context:** Conversational AI agents need context to provide personalized, relevant responses. Three types of context are available: (1) user profile (name, role, hardware tier, preferences), (2) page context (current page URL, title, headings, description), and (3) conversation history (previous messages). Multiple approaches considered: (1) pass context as separate messages, (2) include in system prompt, (3) use agent memory/state. System prompt inclusion chosen (CarFixer pattern) for simplicity and reliability. Context must be extracted client-side (page info), transmitted via request metadata, extracted in backend, and included in agent prompt.
+
+<!-- Significance checklist (ALL must be true to justify this ADR)
+     1) Impact: ✅ Defines how agents receive context, affects personalization and relevance of all responses
+     2) Alternatives: ✅ Separate messages vs system prompt, client-side vs server-side extraction, different transmission methods
+     3) Scope: ✅ Cross-cutting pattern affecting frontend, backend, and agent integration
+-->
+
+## Decision
+
+Use **Context Injection via System Prompt** pattern with the following components:
+
+- **History Pattern**: Include conversation history as formatted string in system prompt (CarFixer pattern), load last 10 messages from thread
+- **User Context**: Extract user profile from authenticated session, include in request metadata, extract in backend, format as string in prompt
+- **Page Context**: Extract page metadata client-side (URL, title, headings, meta tags), include in request metadata, extract in backend, format as string in prompt
+- **Prompt Construction**: Combine history + user context + page context + system prompt into single prompt string
+- **Transmission Method**: Request metadata (JSON in ChatKit protocol body) for user/page context, database query for history
+
+## Consequences
+
+### Positive
+
+- **Simplicity**: Single prompt with all context, no complex state management
+- **Proven Pattern**: Matches CarFixer implementation, works reliably
+- **Agent SDK Compatibility**: Works with current OpenAI Agents SDK version
+- **Client-Side Rich Context**: Can extract detailed page metadata (headings, meta tags) that server can't access
+- **No Extra Requests**: Context included in same request, no additional round-trips
+- **Framework Agnostic**: Pattern works with any agent framework that accepts system prompts
+
+### Negative
+
+- **Prompt Length Growth**: Prompt length increases with history (mitigated by limit=10 messages)
+- **String Formatting**: Must format context as strings, less structured than separate parameters
+- **Client Dependency**: Page context only available if client sends it (could be spoofed)
+- **No Incremental Updates**: Full context rebuilt on each request (not incremental)
+- **Token Usage**: Context included in every request, increases token usage
+
+## Alternatives Considered
+
+**Alternative Pattern A: Pass Context as Separate Messages**
+- Include user/page context as separate user messages in conversation
+- **Why rejected**: Pollutes conversation history, context appears as user messages, less clean
+
+**Alternative Pattern B: Agent Memory/State**
+- Use agent SDK's built-in memory/state management for context
+- **Why rejected**: Not available in current SDK version, would require SDK updates
+
+**Alternative Pattern C: Server-Side Page Context Detection**
+- Detect page context server-side using URL or referrer header
+- **Why rejected**: Server can't access DOM (headings, meta tags), less rich context
+
+**Alternative Pattern D: Context in Query Parameters**
+- Pass context via URL query parameters instead of request body
+- **Why rejected**: Exposes context in URLs/logs, less secure, doesn't work with ChatKit protocol
+
+**Alternative Pattern E: Incremental Context Updates**
+- Only send changed context, maintain context state server-side
+- **Why rejected**: More complex state management, requires context diffing, not needed for current use case
+
+## References
+
+- Feature Spec: `specs/007-chatkit-server/spec.md` (Requirement 5: User Context Injection, Requirement 7: Conversation History Management)
+- Feature Spec: `specs/008-chatkit-ui-widget/spec.md` (Requirement 4: Page Context Extraction, Requirement 6: User Profile Context Transmission)
+- Implementation Plan: `specs/007-chatkit-server/plan.md` (Pattern 2: Context Injection, Pattern 3: History in Prompt)
+- Related ADRs: ADR-0005 (ChatKit Framework Integration Architecture)
+- Evaluator Evidence: `history/prompts/007-chatkit-server/0001-chatkit-server-reverse-engineering.spec.prompt.md`, `history/prompts/008-chatkit-ui-widget/0001-chatkit-ui-widget-reverse-engineering.spec.prompt.md`

history/prompts/007-chatkit-server/0001-chatkit-server-reverse-engineering.spec.prompt.md

@@ -0,0 +1,77 @@
+---
+id: 0001
+title: chatkit-server-reverse-engineering
+stage: spec
+date: 2025-01-27
+surface: claude-code
+model: opus-4-5
+feature: 007-chatkit-server
+branch: 007-chatkit-server
+user: mjs
+command: /sp.reverse-engineer
+labels: [chatkit, backend, specification, reverse-engineering, platform]
+links:
+  spec: specs/007-chatkit-server/spec.md
+  ticket: null
+  adr: history/adr/0005-chatkit-framework-integration-architecture.md
+  pr: null
+files:
+  - specs/007-chatkit-server/spec.md
+  - specs/007-chatkit-server/plan.md
+  - specs/007-chatkit-server/tasks.md
+tests:
+  - null
+---
+
+## Prompt
+
+In this session you lenred chatkit for servrr and ui so i want you to 1. reverse engineer and add 2 full specs of chatkit server and for chatkit ui here: @specs using @.claude/commands/sp.reverse-engineer.md 2. harvest reusbale skills and sybagents using @session-intelligence-harvester to add here @agents @skills
+
+## Response snapshot
+
+Created comprehensive reverse-engineered specification for ChatKit Server:
+
+**Key Deliverables**:
+1. Complete specification at `specs/007-chatkit-server/spec.md` (601 lines) with:
+   - Problem statement and system intent
+   - 8 functional requirements (endpoint handler, thread management, message persistence, agent response, context injection, attachments, history, error handling)
+   - Non-functional requirements (performance, security, reliability, scalability, observability)
+   - System constraints and non-goals
+   - Known gaps and technical debt
+   - Success criteria and acceptance tests
+   - 4 architecture decisions (ADR-001 through ADR-004)
+
+2. Implementation plan at `specs/007-chatkit-server/plan.md` (429 lines) with:
+   - Architecture overview (layered architecture diagram)
+   - 4-layer structure (HTTP/API, ChatKit Server, Agent Execution, Data/Persistence)
+   - 5 design patterns applied
+   - Data flow diagrams
+   - Technology stack decisions
+   - Module breakdown
+   - Regeneration strategy
+
+3. Task breakdown at `specs/007-chatkit-server/tasks.md` (361 lines) with:
+   - 11 phases (Core Infrastructure, Database Layer, ChatKit Server Integration, Agent Integration, HTTP Endpoint, Application Lifecycle, Testing, Improvements, Deployment, Post-Launch)
+   - 23-day timeline estimate
+   - Detailed task breakdown with acceptance criteria
+
+**Reverse Engineering Approach**:
+- Analyzed existing implementation: `rag-agent/chatkit_server.py`, `rag-agent/app.py`, `rag-agent/lifespan.py`
+- Extracted patterns: Framework extension, context injection, history in prompt, attachment converter, graceful degradation
+- Documented architecture decisions with rationale and evidence
+- Identified gaps: Rate limiting missing, auth validation missing, error recovery limited, no monitoring
+
+## Outcome
+
+- ✅ Impact: Complete specification enables regeneration of ChatKit server from scratch or adaptation to other projects. All architectural decisions documented with rationale and evidence.
+- 🧪 Tests: Acceptance tests defined in spec (6 test scenarios covering thread creation, history persistence, context injection, page awareness, attachments, error handling)
+- 📁 Files: Created 3 files (spec.md, plan.md, tasks.md) totaling ~1,400 lines of documentation
+- 🔁 Next prompts: Create PHR for UI widget reverse engineering, create ADRs for architectural decisions, harvest reusable intelligence
+- 🧠 Reflection: Reverse engineering revealed key patterns: (1) Extend ChatKit framework don't replace, (2) Include history in prompt (CarFixer pattern), (3) Context injection via metadata, (4) Graceful degradation for dependencies. These patterns are now reusable across projects.
+
+## Evaluation notes (flywheel)
+
+- Failure modes observed: None - reverse engineering completed successfully with complete documentation
+- Graders run and results (PASS/FAIL): PASS - Specification includes all required sections, architecture decisions documented, acceptance tests defined
+- Prompt variant (if applicable): null
+- Next experiment (smallest change to try): Apply same reverse engineering approach to other implemented features to build comprehensive specification library

history/prompts/007-chatkit-server/0002-chatkit-integration-intelligence-harvesting.misc.prompt.md

@@ -0,0 +1,78 @@
+---
+id: 0002
+title: chatkit-integration-intelligence-harvesting
+stage: misc
+date: 2025-01-27
+surface: claude-code
+model: opus-4-5
+feature: 007-chatkit-server
+branch: 007-chatkit-server
+user: mjs
+command: manual
+labels: [chatkit, intelligence-harvesting, skills, agents, reusable-patterns]
+links:
+  spec: specs/007-chatkit-server/spec.md
+  ticket: null
+  adr: history/adr/0005-chatkit-framework-integration-architecture.md, history/adr/0006-context-injection-pattern-for-conversational-ai.md
+  pr: null
+files:
+  - .claude/skills/engineering/chatkit-integration/SKILL.md
+  - .claude/agents/engineering/chatkit-integration-agent.md
+tests:
+  - null
+---
+
+## Prompt
+
+In this session you lenred chatkit for servrr and ui so i want you to 1. reverse engineer and add 2 full specs of chatkit server and for chatkit ui here: @specs using @.claude/commands/sp.reverse-engineer.md 2. harvest reusbale skills and sybagents using @session-intelligence-harvester to add here @agents @skills
+
+## Response snapshot
+
+Harvested reusable intelligence from ChatKit integration session:
+
+**Key Deliverables**:
+1. Created skill at `.claude/skills/engineering/chatkit-integration/SKILL.md` (428 lines) with:
+   - Persona: Full-stack engineer integrating ChatKit with custom backend
+   - 4 question categories (Backend Integration, Frontend Integration, Context Requirements, Persistence Requirements)
+   - 10 principles (5 backend, 5 frontend) with rationale
+   - 5 implementation patterns with complete code examples:
+     - Pattern 1: ChatKit Server with Custom Agent
+     - Pattern 2: Custom Fetch Interceptor
+     - Pattern 3: Script Loading Detection
+     - Pattern 4: Page Context Extraction
+     - Pattern 5: Text Selection "Ask" Feature
+   - When to apply / contraindications
+   - Common pitfalls and fixes
+   - References to specs and implementation
+
+2. Created agent at `.claude/agents/engineering/chatkit-integration-agent.md` (183 lines) with:
+   - Purpose and capabilities (Backend Integration, Frontend Integration, Context Management, Authentication Integration)
+   - 3-phase workflow (Context Gathering, Backend Setup, Frontend Setup, Integration Testing)
+   - 3 convergence patterns with fixes:
+     - Skipping Context Injection
+     - History Not Remembered
+     - Script Loading Race Condition
+   - Self-monitoring checklist (12 items)
+   - Skills used and references
+
+**Intelligence Harvesting Approach**:
+- Analyzed patterns from reverse engineering session
+- Extracted reusable patterns into skill format (Persona + Questions + Principles)
+- Documented convergence patterns (common mistakes) with fixes
+- Created agent workflow for future ChatKit integrations
+- Made patterns framework-agnostic where possible
+
+## Outcome
+
+- ✅ Impact: Reusable intelligence enables ChatKit integration in any project. Skill provides patterns, agent provides workflow. Both are framework-agnostic and adaptable.
+- 🧪 Tests: Self-monitoring checklist in agent (12 items) ensures quality. Common pitfalls documented with fixes.
+- 📁 Files: Created 2 files (SKILL.md, chatkit-integration-agent.md) totaling ~600 lines of reusable intelligence
+- 🔁 Next prompts: Use chatkit-integration skill in new projects, invoke chatkit-integration-agent for future integrations
+- 🧠 Reflection: Intelligence harvesting revealed that patterns are more valuable than implementation details. Skill focuses on "how to think about ChatKit integration" rather than "copy this code". Agent provides workflow to prevent convergence patterns.
+
+## Evaluation notes (flywheel)
+
+- Failure modes observed: None - intelligence harvesting completed successfully with reusable artifacts
+- Graders run and results (PASS/FAIL): PASS - Skill follows Persona + Questions + Principles format, agent includes workflow and convergence patterns, both are framework-agnostic
+- Prompt variant (if applicable): null
+- Next experiment (smallest change to try): Apply intelligence harvesting to other implemented features to build comprehensive skill/agent library