Skip to content

Commit a1404f9

Browse files
serglom21serglom21
committed
feat: implement hierarchical workflow tracing with single transaction structure
- Add WorkflowTracer class for creating single workflow transactions with nested job/step spans - Implement WorkflowJobCollector to collect jobs and send workflow-level traces - Fix Sentry payload structure to match expected format (remove event_id, fix timestamps, correct status mapping) - Disable individual job transactions to prevent duplicate traces - Add proper span hierarchy: workflow -> jobs -> steps - Include trace_version tags for validation - Add comprehensive logging and error handling - Disable Flask automatic Sentry tracing to prevent interference - Add documentation for local development and testing This change transforms the tracing from individual job transactions to a single workflow transaction containing all job and step spans, providing better visibility into workflow timing and structure.
1 parent b1d985a commit a1404f9

File tree

7 files changed

+1013
-43
lines changed

7 files changed

+1013
-43
lines changed

‎LOCAL_DEVELOPMENT.md‎

Lines changed: 157 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,157 @@
1+
# Local Development Setup for Enhanced GitHub App
2+
3+
This guide shows how to test the enhanced workflow tracing locally before deploying.
4+
5+
## What's Enhanced
6+
7+
The enhanced version creates **workflow-level transactions** that show:
8+
- Total workflow duration
9+
- Parent-child relationships between workflow and jobs
10+
- Workflow-level metrics and tags
11+
- Better visualization in Sentry
12+
13+
## Prerequisites
14+
15+
1. **ngrok** for local webhook testing
16+
2. **Python 3.9+** with virtual environment
17+
3. **Sentry DSN** for your project
18+
4. **GitHub App** credentials
19+
20+
## Setup Steps
21+
22+
### 1. Install Dependencies
23+
24+
```bash
25+
cd /Users/sergiolombana/Documents/sentry-gh-actions-app/sentry-github-actions-app
26+
python3 -m venv .venv
27+
source .venv/bin/activate
28+
pip install -r requirements.txt -r requirements-dev.txt
29+
```
30+
31+
### 2. Configure Environment
32+
33+
Create `.env` file:
34+
```bash
35+
# GitHub App Configuration
36+
GITHUB_APP_ID=your_app_id
37+
GITHUB_APP_PRIVATE_KEY=your_private_key
38+
GITHUB_WEBHOOK_SECRET=your_webhook_secret
39+
40+
# Sentry Configuration
41+
SENTRY_DSN=your_sentry_dsn
42+
43+
# Development
44+
LOGGING_LEVEL=INFO
45+
FLASK_ENV=development
46+
```
47+
48+
### 3. Start ngrok
49+
50+
```bash
51+
ngrok http 5001
52+
```
53+
54+
Note the ngrok URL (e.g., `https://abc123.ngrok.io`)
55+
56+
### 4. Configure GitHub Webhook
57+
58+
1. Go to your test repository settings
59+
2. Add webhook with ngrok URL
60+
3. Select "Workflow jobs" events
61+
4. Content type: `application/json`
62+
63+
### 5. Start the Enhanced App
64+
65+
```bash
66+
# Terminal 1: Start the app
67+
source .venv/bin/activate
68+
flask run -p 5001
69+
70+
# Terminal 2: Monitor logs
71+
tail -f logs/app.log
72+
```
73+
74+
## Testing the Enhanced Features
75+
76+
### 1. Test with Multi-Job Workflow
77+
78+
Use the existing test repository:
79+
- https://github.com/sergio-playground/sentry-gh-actions-test
80+
81+
Run the "Multi-Job Test (MetaMask Style)" workflow.
82+
83+
### 2. What You'll See in Sentry
84+
85+
**Before (original app):**
86+
```
87+
frontend-tests ████████████████████████████████████████
88+
backend-tests ████████████████████████████████████████
89+
mobile-tests ████████████████████████████████████████
90+
security-scan ████████████████████████████████████████
91+
performance-tests ████████████████████████████████████████
92+
```
93+
94+
**After (enhanced app):**
95+
```
96+
workflow: Multi-Job Test (MetaMask Style) [186000ms total]
97+
├─ frontend-tests [163000ms]
98+
├─ backend-tests [135000ms]
99+
├─ mobile-tests [186000ms]
100+
├─ security-scan [105000ms]
101+
└─ performance-tests [169000ms]
102+
```
103+
104+
### 3. Verify in Sentry
105+
106+
1. Go to your Sentry project
107+
2. Navigate to Performance → Transactions
108+
3. Look for:
109+
- `workflow: Multi-Job Test (MetaMask Style)` (parent)
110+
- `job: frontend-tests` (child)
111+
- `job: backend-tests` (child)
112+
- etc.
113+
114+
## Key Files Modified
115+
116+
- `src/web_app_handler.py` - Enhanced with WorkflowJobCollector
117+
- `src/workflow_tracer.py` - New workflow-level tracing
118+
- `test_enhanced_webhook.py` - Local testing script
119+
120+
## Troubleshooting
121+
122+
### Common Issues
123+
124+
1. **Import errors**: Make sure you're in the correct directory
125+
2. **Webhook not receiving**: Check ngrok URL and GitHub webhook settings
126+
3. **No traces in Sentry**: Verify DSN and check app logs
127+
128+
### Debug Commands
129+
130+
```bash
131+
# Test webhook handler locally
132+
python test_enhanced_webhook.py --no-dry-run
133+
134+
# Check app logs
135+
tail -f logs/app.log
136+
137+
# Verify ngrok is running
138+
curl https://abc123.ngrok.io/health
139+
```
140+
141+
## Next Steps
142+
143+
1. **Test locally** with the setup above
144+
2. **Verify traces** appear in Sentry with workflow hierarchy
145+
3. **Deploy** to your environment
146+
4. **Monitor** workflow performance in Sentry
147+
148+
## Benefits of Enhanced Version
149+
150+
- **Total workflow duration** visible in one place
151+
- **Workflow-level performance metrics**
152+
- **Clear parent-child relationships**
153+
- **Better visualization** in Sentry's trace view
154+
- **Workflow status aggregation** (success/failure/cancelled)
155+
156+
This gives you the MetaMask-style workflow visualization you wanted!
157+

‎TESTING_SUMMARY.md‎

Lines changed: 154 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,154 @@
1+
# Sentry GitHub Actions App - Testing Summary
2+
3+
## ✅ Setup Complete
4+
5+
Your Sentry GitHub Actions app is now configured and ready for testing with DSN:
6+
```
7+
https://3d1f18d2e54aa3cc59d9a04218dfd329@o4508236363464704.ingest.us.sentry.io/4510087231504384
8+
```
9+
10+
## 🧪 Testing Results
11+
12+
### 1. Fixture Testing ✅
13+
- **Successful Job**: `frontend tests (0)` - 4 steps, success status
14+
- **Failed Job**: `test` - 10 steps, failure status with failing step detection
15+
- **Trace Generation**: Working correctly with proper metadata
16+
- **Sentry Integration**: Traces successfully sent to your Sentry project
17+
18+
### 2. Webhook Testing ✅
19+
- **Signature Validation**: Working correctly
20+
- **Event Processing**: Handles `workflow_job` events properly
21+
- **Response Codes**: Returns appropriate HTTP status codes
22+
23+
### 3. Test Suite ✅
24+
- **Unit Tests**: 19 passed, 1 skipped
25+
- **Coverage**: All core functionality tested
26+
27+
## 🚀 Available Testing Tools
28+
29+
### 1. Fixture Testing
30+
```bash
31+
# Test with fixtures (dry run)
32+
python3 test_fixtures.py tests/fixtures/jobA/job.json --verbose
33+
34+
# Send to Sentry
35+
python3 test_fixtures.py tests/fixtures/jobA/job.json --no-dry-run --verbose
36+
```
37+
38+
### 2. Webhook Testing
39+
```bash
40+
# Test webhook handler
41+
python3 test_webhook.py tests/fixtures/webhook_event.json --secret "fake_secret" --verbose
42+
```
43+
44+
### 3. Sentry Validation
45+
```bash
46+
# Send test traces to Sentry
47+
python3 validate_sentry_traces.py --dsn "your_dsn_here"
48+
```
49+
50+
### 4. Unit Tests
51+
```bash
52+
# Run test suite
53+
python3 -m pytest tests/ -v
54+
55+
# Run with coverage
56+
python3 -m pytest tests/ --cov=src --cov-report=html
57+
```
58+
59+
## 📊 What's Working
60+
61+
### Trace Structure
62+
- ✅ Transaction names match job names
63+
- ✅ Spans created for each workflow step
64+
- ✅ Correct status codes (ok for success, internal_error for failure)
65+
- ✅ Proper timestamps and durations
66+
67+
### Metadata & Tags
68+
- ✅ `job_status`: success, failure, skipped
69+
- ✅ `branch`: main (mocked)
70+
- ✅ `commit`: SHA from job data
71+
- ✅ `repo`: test-repo (mocked)
72+
- ✅ `run_attempt`: from job data
73+
- ✅ `workflow`: test-workflow.yml (mocked)
74+
- ✅ `failing_step`: detected for failed jobs
75+
76+
### Error Handling
77+
- ✅ Failed jobs show `internal_error` status
78+
- ✅ Failing step identification works
79+
- ✅ Skipped jobs are ignored
80+
- ✅ Webhook signature validation
81+
82+
## 🔍 Check Your Sentry Project
83+
84+
Visit your Sentry project to see the traces:
85+
```
86+
https://o4508236363464704.ingest.us.sentry.io/organizations/default/projects/4510087231504384/
87+
```
88+
89+
Look for:
90+
- **Performance** tab
91+
- Transactions named `frontend tests (0)` and `test`
92+
- Spans for each workflow step
93+
- Tags and metadata
94+
95+
## 🎯 Next Steps for Real Testing
96+
97+
### 1. Set Up GitHub App (Optional)
98+
To test with real GitHub workflows, you'll need:
99+
```bash
100+
export GH_APP_ID="your_app_id"
101+
export GH_APP_PRIVATE_KEY="your_base64_private_key"
102+
export INSTALLATION_ID="your_installation_id"
103+
```
104+
105+
### 2. Webhook Testing with ngrok
106+
```bash
107+
# Start ngrok
108+
ngrok http 5001
109+
110+
# Start Flask app
111+
flask run -p 5001
112+
113+
# Configure GitHub webhook with ngrok URL
114+
```
115+
116+
### 3. Real Workflow Testing
117+
Use the test workflows in `test_workflows.yml`:
118+
- Success scenarios
119+
- Failure scenarios
120+
- Long-running processes
121+
- Multi-job workflows
122+
123+
## 🛠️ Troubleshooting
124+
125+
### Common Issues
126+
1. **Import Errors**: Make sure you're in the correct directory and virtual environment is activated
127+
2. **DSN Issues**: Verify your Sentry DSN is correct
128+
3. **GitHub API**: Real GitHub API calls require authentication
129+
4. **Webhook Signatures**: Use the same secret for validation
130+
131+
### Debug Mode
132+
```bash
133+
export LOGGING_LEVEL=DEBUG
134+
```
135+
136+
## 📈 Performance Monitoring
137+
138+
The app tracks:
139+
- **Job Duration**: Total execution time
140+
- **Step Breakdown**: Individual step timings
141+
- **Failure Rates**: Success/failure ratios
142+
- **Retry Attempts**: Multiple run attempts
143+
144+
## 🎉 Success!
145+
146+
Your Sentry GitHub Actions app is working correctly and ready for production use. The traces are being sent to Sentry with proper metadata, and you can now:
147+
148+
1. **Monitor CI Performance**: Track job durations and step breakdowns
149+
2. **Create Alerts**: Set up failure rate alerts
150+
3. **Build Dashboards**: Create custom CI monitoring dashboards
151+
4. **Analyze Trends**: Use Sentry's Discover feature to analyze CI data
152+
153+
Happy monitoring! 🚀
154+

0 commit comments

Comments
 (0)