Feature/drc 2089 integration change to use the claude agent sdk to replace

[iamcxa]

Migrate to Claude Agent SDK Multi-Agent Architecture

Overview

Refactors the PR analysis agent from custom orchestration to Claude Agent SDK with a multi-agent delegation pattern.

Architecture

Main Orchestrator Agent
├─→ github-context       (fetches PR metadata & file changes)
├─→ recce-analysis       (analyzes dbt model changes via Recce MCP)
└─→ preset-check-executor (runs validation checks from recce.yml)

Key Changes:

• Native delegation: Uses Claude SDK's agents config instead of custom tool routing
• Tool isolation: Each subagent has explicit tool permissions (no wildcards)
• MCP integration: Connects to Recce MCP HTTP server for dbt validation
• Provider abstraction: Supports GitHub/GitLab/Bitbucket with unified interface

Dependencies

⚠️ Critical: Requires Recce MCP HTTP server from DataRecce/recce#932

This PR adds the MCP HTTP server that recce-summary-agent connects to via SSE transport.

Usage

1. Configure Environment

cp .env.example .env
# Edit .env with your tokens:
# - GITHUB_TOKEN=ghp_...
# - ANTHROPIC_API_KEY=sk-ant-...

2. Start Recce MCP Server

Replace /path/to/dbt/project with your dbt project path:

python -m recce.cli mcp-server-http \
  --project-dir /path/to/dbt/project \
  --profiles-dir /path/to/dbt/project \
  --target-path /path/to/dbt/project/target \
  --target-base-path /path/to/dbt/project/target-base

Example with actual path:

python -m recce.cli mcp-server-http \
  --project-dir /recce/jaffle_shop_agentic \
  --profiles-dir /recce/jaffle_shop_agentic \
  --target-path/recce/jaffle_shop_agentic/target \
  --target-base-path /recce/jaffle_shop_agentic/target-base

Server starts at http://0.0.0.0:8080 with SSE endpoint at /sse.

3. Run Agent

pnpm summary <owner> <repo> <pr_number>

Example:

pnpm summary DataRecce jaffle_shop_agentic 3

File Structure

src/
├── agent.ts                 # Main orchestrator (query + subagents)
├── prompts/
│   ├── index.ts            # PromptBuilder with subagent configs
│   └── fragments/          # Modular prompt components
│       ├── base.ts
│       ├── github_context.ts
│       ├── recce_analysis.ts
│       ├── preset_checks.ts
│       └── pr_summary_format.ts
├── providers/              # GitHub/GitLab/Bitbucket abstraction
├── recce/                  # Preset check parsing (recce.yml)
└── templates/              # Output formatters (markdown/json/slack)

Changes Summary

• 49 files changed: +8770 / -566 lines
• Removed: Custom agent core (src/core/agent_core.ts), old prompt system
• Added: Claude SDK integration, subagent prompts, provider abstraction
• Enhanced: Structured logging, preset check execution, error handling

---

Status: ✅ Ready for review
Build: dist/index.js 847.5kb
Dependencies: Requires DataRecce/recce#932 (MCP HTTP server)

[iamcxa]

---

PR Analysis Summary: DataRecce/jaffle_shop_agentic #3

Status: ⚠️ VALIDATION BLOCKED - Environment Configuration Issues

---

PR Overview

Field	Value
PR Title	[bad example] fac_orders
PR Number	#3
Author	iamcxa
Status	Open
Created	October 29, 2025
Last Updated	November 5, 2025
Branch	bad-example-fac_order → main
Link	GitHub PR #3

File Changes Summary

File	Status	Changes
models/customers.sql	Modified	+1 / -1
models/fac_orders.sql	Added	+29 / -0
package-lock.yml	Added	+8 / -0
Total	—	+38 / -1

---

dbt Model Changes

Added Models

• fac_orders (new fact table)
  • Materialized as table
  • Joins stg_orders, stg_customers, and stg_payments
  • Enriches orders with customer names and payment amounts
  • Creates derived fields: payment_count, total_amount

Modified Models

• customers (table)
  • Change: Updated source reference from stg_orders to fac_orders
  • Impact: Now depends on the new fac_orders fact table instead of staging orders
  • Downstream Effects: Cascades to dependent models:
    • customer_segments (impacted)
    • customer_order_pattern (impacted)

Removed Models

None

Lineage Impact

Dependency Chain:

stg_orders, stg_customers, stg_payments
           ↓
        fac_orders (NEW)
           ↓
        customers (MODIFIED) ← Previously sourced from stg_orders
           ↓
    ├── customer_segments (IMPACTED)
    └── customer_order_pattern (IMPACTED)

Critical Observation: The customers model now depends on fac_orders, creating a circular dependency concern:

• customers used to depend on stg_orders (staging layer)
• customers now depends on fac_orders (fact layer)
• This represents a layer inversion that may violate dbt model architecture patterns

Schema Changes

✅ No breaking schema changes detected for customers, orders, or modified nodes

---

Preset Check Results

Overall Status: ❌ FAIL (Validation Blocked)

Check #	Name	Type	Status	Finding
1	Schema of customers/orders	schema_diff	✅ PASS	No breaking changes detected
2	Row count consistency	row_count_diff	❌ ERROR	DB connectivity issue
3	Value diff (CLV)	profile_diff	⚠️ WARN	No profile data available
4	Avg CLV by week	query_diff	❌ ERROR	DB connectivity issue

Summary:

• ✅ Passed: 1 / 4
• ⚠️ Warnings: 1 / 4
• ❌ Errors: 2 / 4

Check Details

✅ Check 1: Schema of customers, orders and modified nodes

• Type: schema_diff
• Result: PASS
• Details: No column additions, removals, or type changes detected for the customers and orders models or any modified nodes.
• Implication: Schema structure is backward compatible.

❌ Check 2: Row count of customers, orders and modified table models

• Type: row_count_diff
• Result: ERROR
• Error Message: Table with name customers does not exist in 'jaffle_shop'.'prod'
• Root Cause: Base/production environment database connectivity failure
• Impact: Cannot validate that row counts remain consistent between base and current versions
• Action Required: Verify production database connection is configured in Recce

⚠️ Check 3: Value diff of customers (customer_lifetime_value)

• Type: profile_diff
• Result: WARNING
• Finding: No profile metrics retrieved for either base or current environment
• Requirement: Check requires 100% matching of customer_lifetime_value values
• Impact: Cannot validate data integrity at the row level
• Action Required: Ensure profile data is generated; verify customers table exists and is accessible

❌ Check 4: Query diff - Average customer_lifetime_value by week

• Type: query_diff
• Result: ERROR
• Error Message: Table with name customers does not exist in 'jaffle_shop'.'prod'
• Root Cause: Same as Check 2 - production database unavailable
• Impact: Cannot verify that average CLV per week remains stable
• Action Required: Restore production database connectivity

---

Impact Analysis

🚨 Critical Issues

1. Validation Blocked Due to Environment Misconfiguration

   • The Recce validation pipeline cannot access the production/base environment
   • The customers table is missing from 'jaffle_shop'.'prod' schema
   • Blocker Status: Data quality validation cannot proceed until resolved

2. Potential Circular Dependency

   • customers model now depends on fac_orders (fact layer)
   • Previously sourced from stg_orders (staging layer)
   • This represents a layer architecture violation if fact tables should not feed into dimensions
   • Risk: Could cause increased load times and complicated model dependencies

3. Incomplete Payment Data Handling

   • fac_orders creates a hardcoded payment_count = 1 field
   • No validation that each order has exactly one payment
   • Could produce inaccurate financial metrics if orders have multiple/zero payments
   • Risk: Incorrect financial reporting downstream

⚠️ Data Quality Concerns

Concern	Severity	Description
Missing validation	HIGH	Cannot validate row counts and CLV values due to DB errors
Hardcoded payment logic	MEDIUM	payment_count = 1 may not reflect actual data
Layer inversion	MEDIUM	Dimension table sourcing from fact table violates typical architecture
Downstream impact	MEDIUM	Two models (customer_segments, customer_order_pattern) affected by changes

📊 Data Quality Assessment

Current Validation Score: ⚠️ INCOMPLETE (25% Validated)

• 1 check passed out of 4
• 2 critical checks blocked by database errors
• 1 check degraded to warning level

Recommendation: DO NOT MERGE until:

1. Production database connectivity is restored
2. All 4 preset checks return PASS status
3. Layer architecture is reviewed and confirmed acceptable
4. Payment data logic is validated

---

Recommendations

Priority 1: Environmental Issues (BLOCKER)

• Verify production database connection - Recce cannot reach 'jaffle_shop'.'prod' schema
• Check database credentials - Ensure proper access permissions are configured
• Verify table deployment - Confirm customers table exists in production
• Re-run validation - Once connectivity is restored, re-execute all 4 preset checks

Priority 2: Architecture Review

• Review layer hierarchy - Confirm if customers should depend on fac_orders or if this is a modeling error
• Assess downstream impact - Validate customer_segments and customer_order_pattern produce correct outputs with the modified source
• Document design decision - If intentional, document why dimension feeds from fact table

Priority 3: Data Validation

• Validate payment logic - Review why payment_count is hardcoded to 1; verify each order has exactly one payment record
• Test joins - Ensure LEFT JOINs on customers and payments don't introduce unexpected NULL values
• Run dbt tests - Execute existing dbt tests to catch data quality issues before merge

Merge Decision

Current Status: ❌ BLOCKED

• Merge should NOT proceed until validation errors are resolved
• Validation should be fully green (all checks PASS) before approval

---

Summary Table

Aspect	Status	Details
Schema Changes	✅ Safe	No breaking column changes
Data Validation	❌ Blocked	DB connectivity issues prevent validation
Model Changes	⚠️ Review	1 added, 1 modified, 2 impacted downstream
Architecture	⚠️ Review	Potential layer inversion (dimension ← fact)
Overall Status	❌ BLOCKED	Environmental issues + architecture concerns

---

Generated: 2025-11-05
PR Link: DataRecce/jaffle_shop_agentic#3

[iamcxa]

change here to use stdio mode mcp

[iamcxa]

---

PR Analysis Summary: DataRecce/jaffle_shop_agentic #3

PR Overview

PR Title: [bad example] fac_orders
PR Number: #3
Author: iamcxa
Status: 🔵 Open
Created: 2025-10-29 | Last Updated: 2025-11-05
URL: DataRecce/jaffle_shop_agentic#3

Summary

This PR introduces a new fact table fac_orders to enhance the dbt project with payment information, enabling better financial analysis and reporting capabilities. The changes include adding a new model, modifying the customers model to reference the new fact table, and introducing dbt package dependencies.

File Changes

• Total Files Changed: 3
• Additions: 38 lines
• Deletions: 1 line

File	Type	Changes
models/fac_orders.sql	Added	+29 lines (new fact table with multi-table join)
models/customers.sql	Modified	+1/-1 lines (reference update: stg_orders → fac_orders)
package-lock.yml	Added	+8 lines (dbt package dependencies)

CI/CD Status: ⏳ Pending (no checks currently configured)

---

dbt Model Changes

Added Models

• fac_orders (Fact Table)
  • New aggregated fact table combining orders, customers, and payments data
  • Schema: order_id, customer_id, customer_first_name, customer_last_name, order_date, status, payment_count, total_amount
  • Dependencies: stg_orders, stg_customers, stg_payments

Modified Models

• customers (Table)
  • Updated upstream reference from stg_orders to fac_orders
  • Impact: Now depends on the new fact table for order information
  • Risk Level: HIGH - affects 8 downstream models

Downstream Impact Analysis

The customers model is a critical hub in the data lineage affecting 8 total models:

Affected Model	Relationship
fac_orders	Direct dependency (new model)
customer_segments	Depends on customers
customer_order_pattern	Depends on customers
orders	Indirectly impacted
stg_customers	Upstream changes propagate
stg_payments	Upstream changes propagate
stg_orders	Upstream changes propagate
Source: raw_customers, raw_orders, raw_payments	All 3 sources modified

Schema Changes

✅ No column-level schema changes detected across customers, orders, and modified models. Changes appear to be logic-level or data-level modifications rather than structural changes.

Row Count Differences

⚠️ Unable to Validate - Row count comparison blocked due to environment configuration issue (see Preset Checks section).

---

Preset Check Results

Overall Status: ❌ BLOCKED

Summary Table:

Check #	Name	Type	Status	Details
1	Model schema of customers, orders, modified nodes	schema_diff	✅ PASS	No schema changes detected
2	Row count of customers, orders, modified tables	row_count_diff	❌ ERROR	Table not found in environment
3	Value diff of customers (lifetime value)	value_diff	❌ ERROR	Table not found in environment
4	Query diff of avg lifetime value	query_diff	❌ ERROR	Table not found in environment

Score: 1/4 Passed | 3/4 Errors

Detailed Findings

✅ Check 1: Schema Validation - PASS

• Description: Verify customers, orders, and modified models have consistent schemas
• Result: No schema changes detected
• Finding: Schema structures remain stable between base and current environments
• Action: ✅ Approved for this check

❌ Check 2-4: Data Validation Checks - ERROR

All three data validation checks (row counts, value matching, aggregate metrics) failed with the same critical issue:

Error: Catalog Error: Table with name customers does not exist!
Root Cause: The materialized dbt models do not exist in the current database environment (jaffle_shop.prod)

Specific Failures:

• Check 2: Cannot compare row counts for customers and orders tables
• Check 3: Cannot verify 100% value matching for customer_lifetime_value using customer_id as primary key
• Check 4: Cannot calculate and compare weekly averages of customer_lifetime_value

---

Impact Analysis

🚨 Critical Issues

1. Environment Configuration Blocker

• Issue: dbt models have not been materialized in the current environment
• Impact: Cannot execute data quality validation checks
• Severity: 🔴 CRITICAL
• Recommendation: Build dbt models before PR validation

2. High-Risk Model Changes

• Issue: New fac_orders fact table with untested aggregation logic
• Impact: Depends on accuracy of payment amount aggregation and join logic
• Severity: 🟠 HIGH
• Recommendation: Validate fact table row counts and metrics match expectations

3. Upstream Source Changes

• Issue: All three raw sources (raw_customers, raw_orders, raw_payments) have been modified
• Impact: Changes cascade through 8 downstream models
• Severity: 🟠 HIGH
• Recommendation: Comprehensive testing of all impacted models

4. Downstream Cascade Risk

• Issue: customers model change affects customer_segments and customer_order_pattern
• Impact: Aggregation and reporting models may produce incorrect metrics
• Severity: 🟠 HIGH
• Recommendation: Re-validate all dependent model outputs

⚠️ Data Quality Warnings

1. Fact Table Aggregation Logic - No validation that payment_count = 1 is appropriate for all scenarios
2. Left Join Behavior - Unvalidated assumptions about orphaned records in outer joins
3. Source Stability - Three simultaneous source changes increase risk of cascading failures

---

Recommendations

Priority 1: BLOCKER - Environment Setup

# 1. Build all dbt models to materialize tables
dbt build

# 2. Verify database schema and table existence
SELECT * FROM information_schema.tables 
WHERE table_schema = 'prod' 
AND table_name IN ('customers', 'orders', 'fac_orders');

Priority 2: HIGH - New Model Validation

• Validate fac_orders row counts match expected order volume
• Verify payment_count is always 1 (or document exceptions)
• Test total_amount matches actual payment data
• Check for NULL values in customer name fields from left join

Priority 3: HIGH - Regression Testing

• Compare customers table row counts before/after merge
• Validate customer_segments output unchanged
• Verify customer_order_pattern metrics are stable
• Run dbt tests on all modified and dependent models

Priority 4: MEDIUM - Documentation

• Add documentation for new fac_orders model
• Document assumptions about payment aggregation
• Update lineage diagrams
• Add data quality tests to models

---

Merge Recommendation

🚫 DO NOT MERGE - The PR is currently blocked due to environment configuration issues preventing data quality validation.

Before Merge:

1. ✅ Materialize dbt models in current environment
2. ✅ Re-run all 4 preset checks and achieve passing status
3. ✅ Validate new fac_orders fact table output
4. ✅ Confirm no breaking changes in downstream models
5. ✅ Obtain code review approval

Status: Waiting for environment setup and validation