Agent Learning & Skilling Platform — Architecture & Design Decisions

Purpose of this document: This is the single source of truth for the platform's architecture, design decisions, and implementation guidance. It is written to be consumed by both humans and AI coding agents (GitHub Copilot CLI, Copilot Workspace, etc.) so they can implement any part of the system with full context.

1. Product Vision

An agentic learning and skilling platform inspired by Microsoft AI Skills Navigator. Users authenticate via Microsoft Entra ID, interact through Microsoft Teams or M365 Copilot, and receive a personalized, AI-driven learning experience that feels like having a personal mentor — not just a content catalog.

Core User Journey

Login (Entra ID)
    │
    ▼
🧭 Assessment Agent ──► "What's your role? What do you know? What are your goals?"
    │
    ▼
🗺️ Path Builder Agent ──► Generates personalized learning path
    │
    ▼
📚 Learner starts modules (videos, labs, docs)
    │
    ├──► 🎓 Coaching Agent ──► "I'm stuck" → real-time Socratic help
    │
    ├──► 📊 Progress Tracker ──► detects drop-off → proactive nudge
    │
    ├──► 🗺️ Path Builder ──► adapts path based on progress
    │
    ▼
📋 Credential Agent ──► assessment → badge → certificate
    │
    ▼
👥 Team Advisor ──► manager sees team skill map & recommendations

Design Philosophy: Mentor, Not Catalog

The agentic experience must feel intimate and personal. This is the difference between a library and a tutor:

Dimension	Web (Catalog)	Agent (Mentor)
Initiative	User-driven	Agent-driven
Memory	Page views, completions	Struggles, preferences, emotions
Timing	When user opens the app	When the learner needs it
Tone	Formal, generic	Personal, adaptive
Teaching	Content delivery	Socratic dialogue
Adaptation	Filter/sort	Reshape entire path in real-time
Relationship	Transactional	Continuous, compounding
Context	Role-based	Job-based, team-aware

2. High-Level Architecture

┌──────────────────────────────────────────────────────────────────────────────────────┐
│                      AGENT LEARNING & SKILLING PLATFORM                              │
│                                                                                      │
│   CLIENT LAYER                                                                       │
│   ┌───────────────┐    ┌───────────────────┐    ┌───────────────────┐                │
│   │  Teams Bot    │    │  Copilot App      │    │  Web App (React)  │                │
│   │  (M365 SDK)   │    │  (M365 SDK)       │    │  (Next.js)        │                │
│   └───────┬───────┘    └─────────┬─────────┘    └─────────┬─────────┘                │
│           └──────────────────────┼─────────────────────────┘                         │
│                                  ▼                                                   │
│   ┌──────────────────────────────────────────────────────────────────────┐            │
│   │                    ENTRA ID  (Authentication + SSO)                   │            │
│   └──────────────────────────┬───────────────────────────────────────────┘            │
│                              ▼                                                       │
│   INGESTION LAYER                                                                    │
│   ┌──────────────────────────────────────────────────────────────────────┐            │
│   │                API GATEWAY  (Azure API Management)                   │            │
│   │   Auth │ Rate Limit (per-user/org) │ Input Sanitize │ Route         │            │
│   └──────────────────────────┬───────────────────────────────────────────┘            │
│                              ▼                                                       │
│          ┌───────────────────┼───────────────────┐                                   │
│          ▼                   ▼                   ▼                                   │
│   ┌─────────────┐    ┌─────────────┐    ┌──────────────────────┐                    │
│   │  User       │    │  Message    │    │  Scheduler /         │                    │
│   │  Requests   │──► │  Queue      │◄───│  Trigger Engine      │                    │
│   └─────────────┘    │ (Svc Bus)   │    │  (proactive nudges)  │                    │
│                      └──────┬──────┘    └──────────────────────┘                    │
│                             ▼                                                        │
│   AGENT LAYER  ─────────────────────────────────────────────────────────────         │
│   ┌──────────────────────────────────────────────────────────────────────┐           │
│   │     AZURE AI FOUNDRY  (Managed Agent Runtime)                        │           │
│   │                                                                      │           │
│   │  ┌────────────┐ ┌────────────┐ ┌────────────┐ ┌────────────┐       │           │
│   │  │ Assessment │ │Path Builder│ │  Coaching   │ │  Progress  │       │           │
│   │  │   Agent    │ │   Agent    │ │   Agent     │ │  Tracker   │       │           │
│   │  └─────┬──────┘ └─────┬──────┘ └─────┬──────┘ └─────┬──────┘       │           │
│   │  ┌─────┴──────┐ ┌─────┴──────┐                                      │           │
│   │  │ Credential │ │Team Advisor│  ← All prompt-based, gpt-4o          │           │
│   │  │   Agent    │ │   Agent    │  ← Entra auth, Responses API         │           │
│   │  └─────┬──────┘ └─────┬──────┘  ← App Insights tracing             │           │
│   │        └───────┬───────┘                                             │           │
│   │                ▼                                                     │           │
│   │  ┌──────────────────────────────────────────────┐                   │           │
│   │  │  TOOL EXECUTION  (openapi tools)              │                   │           │
│   │  │  Foundry calls Azure Functions HTTP endpoints │                   │           │
│   │  │  via OpenAPI spec — agents are self-contained │                   │           │
│   │  └──────────────────────┬───────────────────────┘                   │           │
│   └─────────────────────────┼────────────────────────────────────────────┘           │
│                             ▼                                                        │
│   TOOL LAYER (Azure Functions — Flex Consumption, HTTP Triggers)                      │
│   ┌──────────────────────────────────────────────────────────────────────┐            │
│   │  POST /api/tools/get-learner-profile                                 │            │
│   │  POST /api/tools/upsert-learner-profile                              │            │
│   │  POST /api/tools/update-skill-gaps                                   │            │
│   │  POST /api/tools/update-preferences                                  │            │
│   │  POST /api/tools/get-learning-records                                │            │
│   │  POST /api/tools/get-learning-record                                 │            │
│   │  POST /api/tools/save-learning-path                                  │            │
│   │  POST /api/tools/update-module-status                                │            │
│   │  POST /api/tools/get-current-module-context                          │            │
│   │  POST /api/tools/record-coaching-note                                │            │
│   │  POST /api/tools/log-emotional-signal                                │            │
│   │  GET  /api/openapi.json  (serves OpenAPI 3.0.1 spec for Foundry)     │            │
│   │                                                                      │            │
│   │  Also: Service Bus triggers for async agent orchestration            │            │
│   │  Also: Timer triggers for proactive scheduler                        │            │
│   └──────────────────────────┬───────────────────────────────────────────┘            │
│                              ▼                                                       │
│   DATA LAYER                                                                         │
│   ┌──────────────────────────────────────────────────────────────────────┐            │
│   │  Cosmos DB │ Redis │ AI Search │ Blob Storage │ Data Explorer        │            │
│   └──────────────────────────────────────────────────────────────────────┘            │
│                              │                                                       │
│   OBSERVABILITY                                                                      │
│   ┌──────────────────────────────────────────────────────────────────────┐            │
│   │  App Insights (Foundry tracing) │ Log Analytics │ Azure Monitor      │            │
│   └──────────────────────────────────────────────────────────────────────┘            │
│                              │                                                       │
│   DELIVERY LAYER                                                                     │
│   ┌──────────────────────────────────────────────────────────────────────┐            │
│   │  Adaptive Cards │ Learning Path UI │ Proactive Nudges │ Email       │            │
│   └──────────────────────────────────────────────────────────────────────┘            │
└──────────────────────────────────────────────────────────────────────────────────────┘

3. Architecture Layers — Detailed

3.1 Client Layer

Client	Technology	Purpose
Teams App	Bot Framework, Adaptive Cards	Chat-based learning, proactive nudges, coaching
Copilot App	M365 Copilot Declarative / Custom Engine Agent	Learning within the Copilot experience
Web App	Next.js 15 (App Router, React 19, TypeScript)	Full portal: dashboards, path visualization, admin, onboarding

Decision: All clients connect through Azure Bot Service / M365 Channel for Teams and Copilot, and directly to the API Gateway for the Web App.

Web App Architecture

The Web App uses Next.js 15 with App Router and is structured into two route groups:

Route Group	Scope	Auth Required	Layout
`(marketing)`	Landing page (`/`)	No	`MarketingNav` — logo + Sign In
`(app)`	Dashboard, paths, credentials, settings, onboarding	Yes	`AppNav` — full nav + user avatar + Sign Out

Authentication: NextAuth.js v5 with Microsoft Entra ID provider (JWT strategy, no DB adapter).

Middleware protects all (app) routes; unauthenticated users redirect to /
Dev credentials provider gated behind AUTH_DEV_MODE=true for local testing
Entra oid + tid claims propagated into JWT/session for user identity

Post-auth routing:

First-time user → /onboarding (assessment chat via ChatPanel)
Returning user → /dashboard

Key pages:

Page	Path	Purpose
Landing	`/`	Marketing — value prop, features, CTA to sign in
Onboarding	`/onboarding`	Assessment chat (first-time users)
Dashboard	`/dashboard`	Stats, active learning paths, streaks
Learning Path	`/path/[pathId]`	Modules list with progress
Module + Coaching	`/path/[pathId]/module/[moduleId]`	Content + AI coaching chat
Credentials	`/credentials`	Earned badges and certificates
Settings	`/settings`	Preferences (pace, format, language)

3.2 Authentication — Microsoft Entra ID

SSO across all clients (Teams, Copilot, Web)
User profile enrichment from Microsoft Graph: role, department, manager, team
Tenant/Org identity for multi-tenancy
Conditional access and license validation

Implementation notes:

Web App uses NextAuth.js v5 (next-auth@beta) with Microsoft Entra ID provider and JWT sessions
Bot Framework handles SSO natively for Teams/Copilot
Store the Entra oid (object ID) as the primary user identifier across all data stores
Entra tid (tenant ID) used for multi-tenant data isolation
Query Microsoft Graph API for org structure: GET /me/manager, GET /me/directReports
API routes (/api/tools/*, /api/agents/*, /api/chat) check session server-side; return 401 if unauthenticated (bypassed in dev mode)

3.3 Ingestion Layer

API Gateway (Azure API Management)

Responsibilities:

Authentication: Validate Entra ID JWT tokens
Rate limiting: Per-user and per-org throttling
Input sanitization: Prompt injection defense
Routing: Direct requests to the correct agent queue

Message Queue (Azure Service Bus)

Responsibilities:

Decouple request intake from agent processing
Backpressure handling during traffic spikes
Priority queues: High (coaching — real-time), Medium (path building), Low (analytics)
Dead letter queue (DLQ): Failed tasks for human review
Per-tenant partitioning: Fairness across organizations

Implementation notes:

Use Service Bus Topics + Subscriptions for routing to different agent types
Topic: agent-requests, Subscriptions: assessment, path-builder, coaching, progress, credential, team-advisor
Message schema:

{
  "messageId": "uuid",
  "tenantId": "org-entra-tenant-id",
  "userId": "user-entra-oid",
  "agentType": "coaching | assessment | path-builder | progress | credential | team-advisor",
  "priority": "high | medium | low",
  "payload": {
    "conversationId": "thread-id",
    "userMessage": "I don't understand embeddings",
    "context": {}
  },
  "metadata": {
    "source": "teams | copilot | web",
    "timestamp": "ISO-8601"
  }
}

Scheduler / Trigger Engine

This is what makes the experience proactive (agent-initiated, not just user-initiated).

Trigger Type	Example	Implementation
Time-based	Monday 9am weekly check-in	Azure Functions Timer Trigger
Event-based	Quiz failed, module completed	Cosmos DB Change Feed → Event Grid
Inactivity	5 days without activity	Timer Trigger + query Progress Store
Progress milestone	80% path complete	Cosmos DB Change Feed
Team assignment	Manager assigns new path	Event Grid from admin action

Implementation notes:

Scheduler writes messages to the same Service Bus queue as user requests
Messages include "trigger": "proactive" so agents know this is agent-initiated
Proactive messages are delivered via Bot Framework Proactive Messaging (Teams) or email

3.4 Agent Layer — Foundry Prompt Agents with Server-Side Tools

All agents are prompt-based agents registered in Azure AI Foundry. They are not containerized applications — Foundry manages the full runtime (model inference, tool calling loop, thread state, auth, scaling, and tracing). Your code only provides stateless tool endpoints via Azure Functions HTTP triggers.

Why Prompt Agents + Server-Side Tools?

Concern	Benefit
Independence	Agents are fully self-contained in Foundry — any client can invoke them without middleware
Scalability	Foundry manages the multi-turn reasoning loop; your Functions only handle short, stateless tool calls (~50-200ms each)
Reusability	Agents can be shared across projects, invoked from Teams, Copilot, web app, or other agents
Observability	Foundry traces every reasoning step, tool call, and response via App Insights
Versioning	Foundry manages agent versions with traffic splitting — no redeployment needed for prompt changes

Agent Inventory

Agent	Foundry ID	Responsibilities	Tools	Memory Store	Trigger
🧭 Assessment	`assessment-agent`	Skill gap analysis, role profiling, benchmarking	`openapi`, `memory_search_preview`	`lp-assessment-memory`	User onboarding, periodic re-assessment
🗺️ Path Builder	`path-builder-agent`	Generate/adapt learning paths, sequence modules, calibrate difficulty	`openapi`, `memory_search_preview`	`lp-pathbuilder-memory`	After assessment, on progress events
🎓 Coaching	`coaching-agent`	Socratic teaching, contextual help, emotional awareness, guided labs	`openapi`, `memory_search_preview`	`lp-coaching-memory`	User asks a question during learning
📊 Progress Tracker	`progress-tracker-agent`	Completion monitoring, drop-off detection, nudges, streak tracking	`openapi`, `memory_search_preview`	`lp-progress-memory`	Scheduled checks, event-based triggers
📋 Credential	`credential-agent`	Quiz generation, grading, badge/certificate issuance, lab validation	`openapi`, `memory_search_preview`	`lp-credential-memory`	Module completion, user requests assessment
👥 Team Advisor	`team-advisor-agent`	Org-wide skill maps, gap analysis, training recommendations	`openapi`, `memory_search_preview`	`lp-teamadvisor-memory`	Manager request, scheduled reports

Agent Definition Model

Each agent is defined by:

Kind: prompt — Foundry manages the LLM and reasoning loop
Model: gpt-4o (Azure OpenAI deployment in the same Foundry project)
Instructions: System prompt stored in /src/Agents/{name}/Prompts/system.v1.md
Tools: openapi type — each agent receives a filtered OpenAPI 3.0.1 spec containing only its relevant operations; Foundry calls your Azure Functions HTTP endpoints directly based on the spec
Memory: memory_search_preview tool — each agent has a dedicated memory store for per-user profile facts and chat summaries; scoped via x-memory-user-id header
Protocol: Responses API (invoke via Foundry MCP agent_invoke or direct REST)
Auth: Entra ID with per-user isolation
Temperature: 0.7 (conversational agents)
Tool endpoint auth: Anonymous (endpoints are on Azure Functions Flex Consumption; secure via APIM in production)

Agent Execution Model — Server-Side Tool Execution via OpenAPI

┌─────────────┐     ┌──────────────────────────────────────────────────────┐
│  Any Client  │     │  AZURE AI FOUNDRY (Managed Runtime)                 │
│  (Teams,     │────▶│                                                     │
│   Copilot,   │     │  1. Receive user input                              │
│   Web App,   │     │  2. Load agent definition (prompt + openapi tools)  │
│   curl)      │     │  3. REASONING LOOP (managed by Foundry):            │
│              │     │     a. LLM reasons over input + conversation        │
│              │     │     b. LLM decides to call a tool (operationId)     │
│              │     │     c. Foundry POSTs to Azure Function endpoint  ───┼──┐
│              │     │     d. Tool result returned to LLM                  │  │
│              │     │     e. Repeat until LLM produces final response     │  │
│              │     │  4. Return response to client                       │  │
│              │◀────│                                                     │  │
└─────────────┘     └──────────────────────────────────────────────────────┘  │
                                                                              │
                    ┌──────────────────────────────────────────────────────┐  │
                    │  AZURE FUNCTIONS (Flex Consumption — Tool Endpoints) │◀─┘
                    │  Host: learning-agents-tools.azurewebsites.net       │
                    │                                                      │
                    │  POST /api/tools/get-learner-profile                  │
                    │  POST /api/tools/upsert-learner-profile               │
                    │  POST /api/tools/update-skill-gaps                    │
                    │  POST /api/tools/update-preferences                   │
                    │  POST /api/tools/get-learning-records                 │
                    │  POST /api/tools/get-learning-record                  │
                    │  POST /api/tools/save-learning-path                   │
                    │  POST /api/tools/update-module-status                 │
                    │  POST /api/tools/get-current-module-context           │
                    │  POST /api/tools/record-coaching-note                 │
                    │  POST /api/tools/log-emotional-signal                 │
                    │  GET  /api/openapi.json  (spec served to Foundry)     │
                    │                                                      │
                    │  All POST, JSON body, anonymous auth, ~50-200ms       │
                    └────────────────────────┬─────────────────────────────┘
                                             │
                    ┌────────────────────────┴─────────────────────────────┐
                    │  DATA LAYER                                          │
                    │  Cosmos DB (serverless, DefaultAzureCredential)       │
                    │  Containers: learner-profiles, learning-records,      │
                    │              tenant-configs                           │
                    └──────────────────────────────────────────────────────┘

Key architectural properties:

Foundry owns the loop — your code never manages multi-turn agent conversations
Tool endpoints are stateless — each HTTP call reads/writes Cosmos DB and returns
Functions scale per-tool-call — not per-conversation (orders of magnitude more efficient)
Any client works — no middleware needed; call the Responses API with an agent_reference
Loop limit: Foundry enforces max tool call iterations to prevent runaway costs

Invoking an Agent (any client)

Via Foundry MCP tool:

agent_invoke(
  projectEndpoint: "https://{account}.services.ai.azure.com/api/projects/{project}",
  agentName: "assessment-agent",
  inputText: "I want to learn about AI agents"
)

Via REST (Responses API):

POST /api/projects/{project}/agents/{agent-name}/responses
Host: {account}.services.ai.azure.com
Authorization: Bearer {entra-token}

{
  "input": "I want to learn about AI agents"
}

Multi-turn conversations use the conversationId returned in the response.

Prompt Versioning

System prompts are stored in the codebase at /src/Agents/{name}/Prompts/system.v1.md. When a prompt changes:

Create a new version file (e.g., system.v2.md)
Update the agent in Foundry with agent_update — this creates a new version automatically
Use Foundry's traffic splitting to canary the new version (e.g., 90% v1 / 10% v2)
Promote to 100% after validation

Agent Memory — Per-User Contextual Memory via Foundry

Each agent has a dedicated memory store that gives it persistent, per-user context across conversations. This is a native Foundry capability — the platform manages memory extraction, embedding, retrieval, and scoping.

Memory Architecture

┌──────────────┐     ┌───────────────────────────────────────────────────────┐
│   Web App    │     │  AZURE AI FOUNDRY                                     │
│              │     │                                                       │
│  NextAuth    │     │  Agent: assessment-agent                              │
│  Session     │     │  Tools: openapi, memory_search_preview                │
│  ─────────── │     │                                                       │
│  user.oid ───┼──┐  │  1. User message arrives                              │
│  user.tid    │  │  │  2. Agent queries memory_search_preview tool          │
│              │  │  │     → scope = x-memory-user-id header value           │
│              │  │  │     → retrieves this user's profile facts + history   │
│              │  │  │  3. LLM reasons with memory + tools + conversation    │
│              │  │  │  4. After response, Foundry auto-extracts new facts   │
│              │  │  │     → stored in user's memory partition                │
└──────────────┘  │  └───────────────────────────────────────────────────────┘
                  │
     HTTP Header  │  ┌───────────────────────────────────────────────────────┐
     ─────────────┘  │  MEMORY STORES (one per agent)                        │
  x-memory-user-id   │                                                       │
  = Entra OID        │  lp-assessment-memory    ← assessment-agent           │
                     │  lp-pathbuilder-memory   ← path-builder-agent         │
                     │  lp-coaching-memory      ← coaching-agent             │
                     │  lp-progress-memory      ← progress-tracker-agent     │
                     │  lp-credential-memory    ← credential-agent           │
                     │  lp-teamadvisor-memory   ← team-advisor-agent         │
                     │                                                       │
                     │  Each store partitioned by userId:                     │
                     │  ┌─ user "abc-123" ─────────────────────────┐         │
                     │  │ Profile: "Senior dev, wants AI skills"   │         │
                     │  │ Summary: last 5 conversations compressed │         │
                     │  └──────────────────────────────────────────┘         │
                     │  ┌─ user "xyz-456" ─────────────────────────┐         │
                     │  │ Profile: "PM, learning Agile at Scale"   │         │
                     │  │ Summary: last 3 conversations compressed │         │
                     │  └──────────────────────────────────────────┘         │
                     └───────────────────────────────────────────────────────┘

Identity Flow — How User ID Reaches Memory

The identity chain ensures each user's memories are completely isolated:

Step	Component	What Happens
1	Entra ID SSO	User authenticates; JWT contains `oid` (Entra object ID) and `tid` (tenant ID)
2	NextAuth callback	`session({ session, token })` maps `token.oid` → `session.user.oid`
3	`buildUserContext()`	Extracts `userId = user.oid \|\| user.id` from session (`src/Clients/WebApp/src/lib/agent-context.ts`)
4	Agent API route	Sends `x-memory-user-id: <userId>` header to Foundry (`src/Clients/WebApp/src/app/api/agents/[agentName]/route.ts`, line 45)
5	Foundry memory tool	Agent's `memory_search` tool configured with `"scope": "{{$userId}}"` — Foundry resolves `{{$userId}}` from the `x-memory-user-id` header
6	Memory store	All reads/writes partitioned to that scope — user A never sees user B's memories

Two Memory Types

Type	Purpose	Behavior
User Profile Memory	Persistent facts about the learner (role, goals, skill level, preferences)	LLM extracts key facts from each conversation; facts accumulate and update over time
Chat Summary Memory	Compressed conversation history for continuity	Foundry summarizes past conversations so the agent can reference prior interactions without loading full chat history

Each agent's memory store has a tailored user_profile_details prompt that guides what facts the LLM should extract. For example:

Assessment Agent → role, experience level, skill gaps, career goals, learning style
Coaching Agent → emotional state, confidence level, topics that resonate, teaching style preference
Progress Tracker → completion patterns, drop-off triggers, preferred study times

Why Per-Agent Memory Stores?

Each agent has its own memory store (not a shared one) for these reasons:

Focused extraction — the assessment agent extracts different facts than the coaching agent; separate stores let each tailor what they remember
No cross-contamination — coaching notes don't pollute the credential agent's context window
Independent scaling — high-traffic agents (coaching) accumulate more memories without affecting low-traffic ones (team-advisor)
Simpler debugging — inspect one agent's memory in isolation

Agent Sync & Memory Provisioning

Memory stores and memory tools are provisioned via the sync script (infra/foundry/sync-agents.sh):

Creates memory stores — one per agent, configured with gpt-4o for extraction and text-embedding-3-small for semantic search
Adds memory_search tool — injected into each agent's tool array alongside existing openapi tools
Preserves existing tools — the script fetches the current agent definition and merges, never overwrites

# Sync all agents (creates memory stores + updates agent versions)
./infra/foundry/sync-agents.sh

# Preview changes without applying
./infra/foundry/sync-agents.sh --dry-run

Memory Store Configuration

Defined in infra/foundry/agents.json per agent:

{
  "memoryStore": {
    "name": "lp-assessment-memory",
    "description": "Assessment agent memory — learner profiles and skill assessments",
    "userProfileDetails": "Extract: current role, experience level, skill gaps..."
  }
}

Memory stores require the Foundry-Features: MemoryStores=V1Preview header (preview feature as of April 2026).

3.5 Data Layer

Data Store Overview

Store	Technology	Purpose	Access Pattern
Operational	Azure Cosmos DB (NoSQL)	Learner profiles, progress, credentials, agent state, org data	Read/Write (hot path)
Cache	Azure Redis Cache	Active sessions, hot profiles, rate limit counters	Read-heavy, short TTL
Search & Retrieval	Azure AI Search (Foundry IQ)	Content catalog, role taxonomies, org-specific content	Read-only (RAG retrieval)
Content Assets	Azure Blob Storage	Videos, documents, lab templates, generated audio (TTS)	Read-heavy (streaming)
Analytics & Audit	Azure Data Explorer	Learning analytics, token costs, audit trail, agent performance	Write-heavy, dashboard queries

Cosmos DB — Data Model

Account: learning-agents-cosmos (serverless, East US 2) Database: learning-platform Auth: DefaultAzureCredential (managed identity with Cosmos DB Built-in Data Contributor role)

Partition strategy: tenantId as partition key for tenant isolation. Use hierarchical partition keys: tenantId / userId where supported.

Learner Profile Container

{
  "id": "learner-{entra-oid}",
  "tenantId": "tenant-id",
  "userId": "entra-oid",
  "type": "learner-profile",

  "cognitiveProfile": {
    "skillGraph": {
      "ai-fundamentals": { "level": 0.7, "confidence": "high", "lastAssessed": "2026-04-10T00:00:00Z" },
      "prompt-engineering": { "level": 0.3, "confidence": "medium", "lastAssessed": "2026-04-12T00:00:00Z" },
      "rag-patterns": { "level": 0.1, "confidence": "low", "lastAssessed": "2026-04-15T00:00:00Z" }
    },
    "weakAreas": ["rag-patterns", "embeddings"],
    "strengths": ["ai-fundamentals", "azure-basics"]
  },

  "preferences": {
    "format": "video",
    "pace": "moderate",
    "preferredTime": "morning",
    "language": "en",
    "learningStyle": "examples-over-theory"
  },

  "behavioralSignals": {
    "avgTimeOnModule": 1200,
    "reReadPatterns": ["embeddings", "vector-search"],
    "quizRetryCount": { "rag-patterns": 2 },
    "sessionFrequency": "3-per-week",
    "lastActiveAt": "2026-04-17T14:30:00Z"
  },

  "emotionalContext": {
    "confidenceLevel": "medium",
    "engagementTrend": "rising",
    "frustrationSignals": ["repeated-quiz-failure-rag"]
  },

  "goals": ["become-ai-engineer", "azure-ai-certification"],
  "role": "Software Engineer",
  "department": "Platform Engineering",
  "managerId": "manager-entra-oid"
}

Learning Record Container

{
  "id": "record-{uuid}",
  "tenantId": "tenant-id",
  "userId": "entra-oid",
  "type": "learning-record",

  "pathId": "path-{uuid}",
  "pathTitle": "AI Engineer Fundamentals",
  "modules": [
    {
      "moduleId": "mod-001",
      "title": "Introduction to AI",
      "status": "completed",
      "score": 92,
      "timeSpentSeconds": 1800,
      "completedAt": "2026-04-11T10:00:00Z",
      "attempts": 1
    },
    {
      "moduleId": "mod-002",
      "title": "Embeddings Deep Dive",
      "status": "in-progress",
      "timeSpentSeconds": 3200,
      "attempts": 2,
      "agentNotes": "Learner struggled with vector similarity — coaching agent switched to analogy-based explanation"
    }
  ],

  "credentials": [
    {
      "credentialId": "cred-001",
      "title": "AI Fundamentals Badge",
      "issuedAt": "2026-04-12T00:00:00Z",
      "type": "badge",
      "verificationUrl": "https://..."
    }
  ],

  "streakDays": 5,
  "totalTimeSpentSeconds": 18000
}

Org / Tenant Container

{
  "id": "tenant-{entra-tenant-id}",
  "tenantId": "tenant-id",
  "type": "tenant-config",

  "orgName": "Contoso",
  "licenseTier": "enterprise",
  "rateLimits": { "requestsPerMinute": 100, "tokensPerDay": 500000 },
  "branding": { "logo": "https://...", "primaryColor": "#0078D4" },
  "dataResidency": "eu-west",
  "assignedPaths": [
    { "pathId": "path-rag-101", "assignedBy": "manager-oid", "assignedTo": ["team-engineering"], "dueDate": "2026-05-01" }
  ],
  "skillHeatMap": {
    "ai-fundamentals": { "avgLevel": 0.6, "coverage": 0.85 },
    "rag-patterns": { "avgLevel": 0.2, "coverage": 0.30 }
  }
}

Redis Cache — Key Patterns

session:{userId}           → Active conversation thread context (TTL: 30 min)
profile:{userId}           → Hot learner profile (TTL: 5 min)
ratelimit:{tenantId}       → Request counter (TTL: 1 min sliding window)
ratelimit:{userId}         → Per-user counter (TTL: 1 min)

Azure AI Search — Foundry IQ Indexes

Index	Contents	Tenant Scoping
`content-catalog`	All courses, videos, labs, docs, podcasts with metadata (difficulty, role, skills, format, duration)	Global (shared content)
`role-taxonomy`	Job roles, skill frameworks, industry benchmarks	Global
`org-content-{tenantId}`	Tenant-specific custom content uploaded by org admins	Per-tenant

Blob Storage — Container Structure

/content
  /videos/{contentId}.mp4
  /documents/{contentId}.pdf
  /labs/{contentId}/
    template.json
    instructions.md
  /generated
    /podcasts/{contentId}-{userId}.mp3    # TTS-generated, cached per learner
    /summaries/{contentId}-simplified.md  # LLM-generated simplified versions

Data Explorer — Analytics Tables

// Learning analytics
LearnerEvents
| where Timestamp > ago(7d)
| summarize CompletionRate = countif(EventType == "module_completed") / count()
  by TenantId, UserId

// Token cost tracking
AgentTokenUsage
| summarize TotalTokens = sum(TokenCount), TotalCost = sum(EstimatedCost)
  by TenantId, AgentType, bin(Timestamp, 1d)

// Audit trail
AuditLog
| where Action == "credential_issued"
| project Timestamp, TenantId, UserId, CredentialId, IssuedBy

Data Flow

Agents ──Read/Write──► Cosmos DB (Operational)
Agents ──Read──────────► AI Search (Retrieval via Foundry IQ)
Agents ──Write─────────► Data Explorer (Telemetry, audit)
Cosmos DB ──Change Feed──► Event Grid ──► Scheduler (triggers)
Cosmos DB ──Change Feed──► Data Explorer (real-time analytics sync)
Agents ──Read/Write──► Redis (session cache, hot profiles)
Agents ──Read──────────► Blob Storage (content assets)
Content Adapter ──Write──► Blob Storage (generated podcasts, summaries)

3.6 Content Adapter Service

Transforms content to match learner preferences on the fly.

Transformation	Input	Output	Technology
Doc → Podcast	PDF/Markdown	MP3 audio	Azure OpenAI (summarize) + Azure Speech (TTS)
Long → Bite-sized	Full module	5-min summary	Azure OpenAI
Complex → Simplified	Advanced content	ELI5 version with analogies	Azure OpenAI
Generic → Role-specific	Generic examples	Examples using learner's domain	Azure OpenAI
Text → Interactive	Static doc	Step-by-step guided lab	Azure OpenAI + Lab template engine

Implementation notes:

Run as Azure Functions (event-driven, scale to zero)
Cache generated content in Blob Storage: /content/generated/
Cache key: {contentId}-{transformationType}-{parameters-hash}
Invalidate cache when source content updates

3.7 Delivery Layer

Channel	Technology	Use Case
Adaptive Cards	Bot Framework + Teams	Rich interactive responses (quiz cards, path cards, progress cards)
Streaming (SSE)	Direct API	Real-time coaching responses in Web App
Proactive Messages	Bot Framework Proactive Messaging	Agent-initiated nudges in Teams/Copilot
Email Digest	Azure Communication Services	Weekly progress summary
Learning Path UI	Web App (React)	Visual path with progress, interactive modules

Proactive messaging implementation:

Store the Teams conversationReference on first user interaction (in Cosmos DB)
Use Bot Framework continueConversation() to send proactive messages
Respect user preferences: opt-out, quiet hours, frequency caps

4. Cross-Cutting Concerns

4.1 Security

Concern	Implementation
Authentication	Entra ID SSO, JWT validation at API Gateway
Identity propagation	Entra `oid` flows from NextAuth session → `buildUserContext()` → `x-memory-user-id` header → Foundry memory scope
Input sanitization	Prompt injection defense at API Gateway layer
Output guardrails	Content Safety API check before delivery; PII redaction
Tool permission scoping	Each agent has a filtered OpenAPI spec with only its relevant operations; endpoints are anonymous (secured by APIM in production)
Data isolation	Cosmos DB partitioned by `tenantId`; AI Search tenant-scoped indexes
Memory isolation	Each Foundry memory store scoped by `userId` via `{{$userId}}` template; users cannot access each other's memories
Audit logging	All agent actions logged to Data Explorer

4.2 Multi-Tenancy

Concern	Implementation
Data isolation	Cosmos DB partition key = `tenantId`
Content scoping	Foundry IQ: global index + per-tenant index
Rate limiting	Per-tenant and per-user limits at API Gateway + Redis counters
Configuration	Tenant config in Cosmos DB (branding, limits, features)
Data residency	Cosmos DB multi-region with geo-fencing per tenant config

4.3 Observability

Deployed resources:

Application Insights: learning-agents-appinsights (connected to Foundry project for agent tracing)
App Insights for Functions: learning-agents-tools (auto-created with Function App)
Log Analytics workspace: learning-agents-logs

Signal	Technology	What to Track
Agent tracing	Application Insights (Foundry integration)	Full reasoning chain: input → tool calls → tool results → output
Tool endpoint metrics	Application Insights (Functions)	Latency per endpoint, error rates, cold starts
Token costs	Custom logging → Data Explorer	Tokens per tenant, per agent, per model, per day
Agent decisions	Structured logging → Data Explorer	Why the agent chose action X (explainability)
Alerting	Azure Monitor	Queue depth, error rates, cost spikes, drop-off spikes

4.4 Cost Management

Strategy	Implementation
Token metering	Log every LLM call with token count; aggregate per tenant in Data Explorer
Model tiering	GPT-4o-mini for simple tasks (nudges, summaries); GPT-4o for complex reasoning (coaching, assessment)
Caching	Cache generated content (podcasts, summaries) in Blob Storage; cache LLM responses for common questions in Redis
Loop limits	Max 10 iterations per agent execution loop
Budget caps	Per-tenant daily token budget; alert at 80%, hard stop at 100%

4.5 Testing & Evaluation

Practice	Implementation
Agent evals	Golden dataset of learner scenarios with expected outcomes; run on every prompt/agent change
Regression tests	CI pipeline compares new agent version outputs against baseline
Shadow mode	New agent versions process prod traffic in parallel without delivery; compare results
Human-in-the-loop	Flag low-confidence agent outputs for human review before delivery
A/B testing	Feature flags to route % of learners to new agent version
Learner feedback	Thumbs up/down on every agent response; stored in Data Explorer

4.6 Versioning & Deployment

Concern	Implementation
Agent versioning	Foundry auto-versions agents (v1, v2, ...) on each update via `POST /agents/{name}/versions`; sync script preserves existing tools when creating new versions
Prompt versioning	Prompts stored as files in `/src/Agents/{name}/Prompts/system.v1.md` with version suffixes
Agent sync	`infra/foundry/sync-agents.sh` — creates/updates agents, provisions memory stores, injects memory tools
Blue/green deploys	Foundry supports traffic splitting across agent versions for canary rollouts
Feature flags	Azure App Configuration for per-tenant feature toggles
Rollback	Revert to previous agent version in Foundry

4.7 Compliance & Governance

Concern	Implementation
Content safety	Azure AI Content Safety API integrated in output guardrails
PII handling	Detect and redact PII in agent inputs/outputs using Presidio or Azure AI Language
Responsible AI	Transparency: learners can ask "why was this recommended?"; fairness: eval for bias across demographics
Accessibility	WCAG 2.1 AA compliance; screen reader support; keyboard navigation; content in multiple formats
GDPR / Privacy	Data export/delete on request; consent management; retention policies in Cosmos DB TTL
Consent	Explicit consent for behavioral tracking; opt-out for proactive messaging

5. Technology Stack Summary

Layer	Technology	Why
Clients	Teams Bot (M365 Agents SDK), M365 Copilot (Declarative Agent), Next.js 15 (App Router, React 19, TypeScript)	Meet learners where they are
Auth	Microsoft Entra ID, NextAuth.js v5 (Web), MSAL (backend)	Enterprise SSO, org graph
API Gateway	Azure API Management	Auth, rate limiting, routing
Queue	Azure Service Bus (Topics + Subscriptions)	Priority queues, tenant fairness, DLQ
Scheduler	Azure Functions Timer + Event Grid	Proactive agent-initiated outreach
Agent Runtime	Azure AI Foundry (prompt agents)	Managed reasoning loop, scaling, versioning, tracing
Agent Tools	Azure Functions (HTTP triggers, Flex Consumption)	Stateless tool endpoints called by Foundry via OpenAPI; auto-scale per-call
Agent Tool Integration	OpenAPI 3.0.1 spec	Each agent receives a filtered spec with only its operations; Foundry resolves operationIds to HTTP calls
Agent Framework	Microsoft Agent Framework 1.0 (C# / .NET 10)	Agent orchestration, multi-agent workflows
LLM	Azure OpenAI (GPT-4o) via Foundry	Reasoning, content generation
Retrieval	Azure AI Search (Foundry IQ)	RAG over content catalog
Operational DB	Azure Cosmos DB (NoSQL)	Profiles, progress, credentials, config
Cache	Azure Redis Cache	Sessions, hot data, rate limits
Content Storage	Azure Blob Storage	Videos, docs, labs, generated audio
Content Transform	Azure Functions + OpenAI + Azure Speech	Doc→podcast, simplify, reformat
Analytics	Azure Data Explorer	Learning analytics, costs, audit
Monitoring	Application Insights + Log Analytics + Azure Monitor	Foundry agent tracing, metrics, alerting
Feature Flags	Azure App Configuration	A/B testing, per-tenant toggles
Email	Azure Communication Services	Weekly digests, notifications

6. Project Structure

/
├── ARCHITECTURE.md              # This file — architectural source of truth
├── AGENTS.md                    # Agent coding conventions & implementation rules
├── LearningPlatform.slnx        # .NET solution file
├── /src
│   ├── /Agents                  # Agent prompt definitions (one per agent)
│   │   ├── /Assessment          # LearningPlatform.Agents.Assessment.csproj
│   │   │   └── Prompts/system.v1.md
│   │   ├── /PathBuilder
│   │   ├── /Coaching
│   │   ├── /ProgressTracker
│   │   ├── /Credential
│   │   └── /TeamAdvisor
│   ├── /Api
│   │   ├── /Gateway             # APIM policies (not .NET)
│   │   └── /Functions           # LearningPlatform.Functions.csproj
│   │       ├── Program.cs          # DI setup, local dev toggle
│   │       ├── openapi.json        # OpenAPI 3.0.1 spec (embedded resource)
│   │       ├── host.json           # Functions runtime config
│   │       ├── /Triggers           # Service Bus triggers (async orchestration)
│   │       └── /Tools              # HTTP triggers (Foundry tool endpoints)
│   │           ├── LearnerProfileEndpoints.cs
│   │           ├── LearningRecordEndpoints.cs
│   │           ├── CoachingEndpoints.cs
│   │           ├── OpenApiEndpoint.cs     # GET /api/openapi.json
│   │           ├── ToolRequestModels.cs
│   │           ├── LearningRecordRequests.cs
│   │           └── CoachingRequests.cs
│   ├── /Clients
│   │   ├── /TeamsBot            # LearningPlatform.Clients.TeamsBot.csproj (M365 Agents SDK)
│   │   ├── /CopilotApp          # LearningPlatform.Clients.CopilotApp.csproj (M365 Agents SDK)
│   │   └── /WebApp              # Next.js / React (TypeScript)
│   └── /Shared
│       ├── /Models              # LearningPlatform.Shared.Models.csproj
│       ├── /Contracts           # LearningPlatform.Shared.Contracts.csproj
│       ├── /Utils               # LearningPlatform.Shared.Utils.csproj
│       ├── /Persistence         # LearningPlatform.Shared.Persistence.csproj (Cosmos DB repos)
│       └── /AgentHost           # LearningPlatform.Shared.AgentHost.csproj (shared agent logic)
│           ├── /Tools           # Tool implementations (called by HTTP endpoints)
│           │   ├── LearnerProfileTools.cs
│           │   ├── LearningRecordTools.cs
│           │   └── CoachingTools.cs
│           └── /Workflows       # Multi-agent workflow orchestration
├── /infra
│   ├── /bicep                   # Azure Bicep templates
│   └── /scripts                 # Deployment scripts
└── /tests
    ├── /AgentEvals              # LearningPlatform.Tests.AgentEvals.csproj
    ├── /Integration             # LearningPlatform.Tests.Integration.csproj
    └── /Load                    # Load testing (k6 or similar)

Key conventions:

Agent projects contain prompt definitions only — the runtime is Foundry
Tool logic lives in Shared/AgentHost/Tools/ — reusable across agents
Tool HTTP endpoints live in Api/Functions/Tools/ — thin wrappers that delegate to tool classes
Shared projects are referenced by Functions, Teams Bot, and Copilot App — never the web app

7. Key Design Decisions Log

#	Decision	Rationale	Alternatives Considered
1	Foundry Prompt Agents over containerized agents	No infrastructure to manage; Foundry handles reasoning loop, scaling, auth, tracing, versioning	Containerized agents on Foundry (more control but more ops), self-hosted on ACA/Functions
2	Server-side tool execution (openapi) over client-side	Agents are fully independent from client code; any client can invoke without middleware; Functions scale per-tool-call not per-conversation	Client-side tool loop (simpler initial setup but creates middleware bottleneck)
3	OpenAPI tool type over azure_function	`openapi` uses direct HTTP calls (fast, ~50-200ms round-trip); `azure_function` is queue-based (input queue → function → output queue with CorrelationId) designed for long-running tasks	`azure_function` queue-based pattern (adds latency and complexity for simple CRUD); `function` type (client-side tool execution)
3	Service Bus over Storage Queue	Priority queues, topics/subscriptions for agent routing, DLQ, sessions	Azure Storage Queue (simpler but fewer features)
4	Cosmos DB as primary operational store	Global distribution, low latency, flexible schema, change feed for events	PostgreSQL (relational but less flexible for profile data)
5	Separate agents per concern vs. monolithic agent	Independent scaling, clearer prompts, easier testing and versioning	Single agent with tool routing (simpler but harder to scale/test)
6	Proactive scheduler as first-class component	Core to the "mentor not catalog" philosophy; without it, agents are reactive only	Rely on user-initiated interactions only
7	Deep learner memory model with behavioral + emotional signals	Enables the intimate, adaptive experience; simple completion tracking is not enough	Basic progress tracking only
8	Content Adapter Service	Learners have different preferences; serving static content breaks the personalization promise	Serve content as-is from catalog
9	Foundry IQ for retrieval	Managed RAG over content catalog; handles indexing, chunking, retrieval	Custom RAG pipeline (more control but more maintenance)
10	Azure Data Explorer for analytics	Optimized for time-series and log analytics; handles high write throughput from telemetry	Log Analytics (simpler but less queryable), Synapse (overkill for this)
11	Multi-client (Teams + Copilot + Web)	Meet learners where they already work; Teams for nudges, Copilot for in-flow, Web for deep sessions	Single client only
12	C# / .NET 10 for all backend	Type safety, Azure SDK first-class support, Microsoft Agent Framework 1.0 native	Python (common for AI but weaker Azure integration), Node.js
13	M365 Agents SDK over Bot Framework SDK	Modern channel integration for Teams/Copilot; replaces deprecated Bot Framework patterns	Bot Framework SDK (legacy, being superseded)
14	Flex Consumption plan over standard Consumption	Supports identity-based storage auth (no shared keys); subscription policy blocks `allowSharedKeyAccess`; better cold-start and scaling	Standard Consumption (blocked by shared key policy), Premium (more expensive)
15	Anonymous auth on tool endpoints	Foundry's OpenAPI tool caller had issues passing API keys correctly; endpoints are fast CRUD operations behind Azure infrastructure; APIM will add auth in production	Function-level API key auth (Foundry 401 errors), Managed Identity (not supported for OpenAPI tool type)
16	DefaultAzureCredential for Cosmos DB	No connection strings to manage; Function App's system-assigned managed identity gets Cosmos DB Built-in Data Contributor role	Connection string (security risk, rotation burden)
17	Cosmos DB serverless over provisioned throughput	Cost-effective for development and bursty workloads; no idle capacity costs	Provisioned RU/s (predictable but wasteful during low-traffic periods)
18	Per-agent filtered OpenAPI spec	Each agent only sees its relevant operations (e.g., coaching-agent gets 5 ops, team-advisor gets 2); reduces confusion and prevents agents from calling tools outside their scope	Single shared spec for all agents (simpler but agents see irrelevant tools)
19	Per-agent memory stores over shared memory	Each agent extracts different facts (assessment: skills, coaching: emotional state); separate stores prevent cross-contamination and enable independent scaling	Single shared memory store (simpler but noisy context, harder to debug)
20	Entra OID as memory scope key	Globally unique, stable per user, already available in JWT; consistent with Cosmos DB userId partitioning	Email (can change), session ID (ephemeral), custom GUID (another mapping to maintain)
21	Foundry native memory over custom RAG	Zero infrastructure — Foundry handles extraction, embedding, retrieval, and scoping; auto-learns from conversations without explicit writes	Custom vector DB + embedding pipeline (full control but significant ops burden)

8. Implementation Priority

Suggested order for incremental delivery:

✅ Foundation: Cosmos DB (serverless, DefaultAzureCredential), Service Bus topics, App Insights
✅ Assessment Agent: Registered in Foundry with OpenAPI tools, end-to-end tested
✅ All 6 Agents: Assessment, Path Builder, Coaching, Progress Tracker, Credential, Team Advisor — all registered with filtered OpenAPI tool specs
✅ Tool Endpoints: 11 HTTP endpoints on Azure Functions (Flex Consumption), OpenAPI 3.0.1 spec served at /api/openapi.json
✅ Cosmos DB Integration: learner-profiles, learning-records, tenant-configs containers; RBAC via managed identity
✅ End-to-end verified: Agent → Foundry reasoning → OpenAPI tool call → Azure Function → Cosmos DB → response
✅ Agent Memory: Per-user memory stores (6 stores), embedding model deployed, memory_search_preview tool on all agents, identity flow wired (Entra OID → x-memory-user-id → Foundry scope)
🔲 Credential Agent tools: Quiz generation, grading, badge/certificate issuance
🔲 Progress Tracker: Drop-off detection, proactive nudges, scheduler
🔲 Onboarding Workflow: Assessment → PathBuilder sequential agent orchestration
🔲 Teams Bot: Delivers assessment + path via Adaptive Cards in Teams
🔲 Web App: Full portal with dashboards and path visualization (Next.js/React)
🔲 APIM Gateway: Auth, rate limiting, routing in front of tool endpoints
🔲 Content Adapter: Format transformation (doc→podcast, simplify)
🔲 Copilot App: M365 Copilot integration
🔲 Analytics & Observability: Full Data Explorer dashboards, cost tracking

This document is the architectural source of truth. Update it as decisions evolve.

FilesExpand file tree

ARCHITECTURE.md

Latest commit

History

ARCHITECTURE.md

File metadata and controls

Agent Learning & Skilling Platform — Architecture & Design Decisions

1. Product Vision

Core User Journey

Design Philosophy: Mentor, Not Catalog

2. High-Level Architecture

3. Architecture Layers — Detailed

3.1 Client Layer

Web App Architecture

3.2 Authentication — Microsoft Entra ID

3.3 Ingestion Layer

API Gateway (Azure API Management)

Message Queue (Azure Service Bus)

Scheduler / Trigger Engine

3.4 Agent Layer — Foundry Prompt Agents with Server-Side Tools

Why Prompt Agents + Server-Side Tools?

Agent Inventory

Agent Definition Model

Agent Execution Model — Server-Side Tool Execution via OpenAPI

Invoking an Agent (any client)

Prompt Versioning

Agent Memory — Per-User Contextual Memory via Foundry

Memory Architecture

Identity Flow — How User ID Reaches Memory

Two Memory Types

Why Per-Agent Memory Stores?

Agent Sync & Memory Provisioning

Memory Store Configuration

3.5 Data Layer

Data Store Overview

Cosmos DB — Data Model

Learner Profile Container

Learning Record Container

Org / Tenant Container

Redis Cache — Key Patterns

Azure AI Search — Foundry IQ Indexes

Blob Storage — Container Structure

Data Explorer — Analytics Tables

Data Flow

3.6 Content Adapter Service

3.7 Delivery Layer

4. Cross-Cutting Concerns

4.1 Security

4.2 Multi-Tenancy

4.3 Observability

4.4 Cost Management

4.5 Testing & Evaluation

4.6 Versioning & Deployment

4.7 Compliance & Governance

5. Technology Stack Summary

6. Project Structure

7. Key Design Decisions Log

8. Implementation Priority