Skip to content

Commit 51d82f1

Browse files
abadawi591abadawi591
committed
refactor: Fetch GitHub PAT from Key Vault instead of ADO pipeline variable
- Added Key Vault token fetching to GitHubClient.__init__() - Token now fetched from mariner-pipelines-kv/cblmarghGithubPRPat using Managed Identity - Removed GITHUB_TOKEN pipeline variable dependency from YAML - Added azure-keyvault-secrets>=4.7.0 to requirements.txt - Single source of truth for PAT (Key Vault only) - Falls back to environment variables for local testing
1 parent d5f98d5 commit 51d82f1

File tree

4 files changed

+303
-30
lines changed

4 files changed

+303
-30
lines changed

‎.pipelines/prchecks/CveSpecFilePRCheck/CveSpecFilePRCheck.yml‎

Lines changed: 2 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -116,22 +116,14 @@ steps:
116116
AZURE_CLIENT_ID: "7bf2e2c3-009a-460e-90d4-eff987a8d71d"
117117
# GitHub integration environment variables
118118
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
119-
# GITHUB_TOKEN: $(cblmarghGithubPRPat)
120-
GITHUB_TOKEN: $(githubPrPat)
119+
# GITHUB_TOKEN removed - now fetched from Key Vault in Python code
121120
GITHUB_REPOSITORY: $(Build.Repository.Name)
122121
GITHUB_PR_NUMBER: $(System.PullRequest.PullRequestNumber)
123122
inputs:
124123
targetType: inline
125124
script: |
126125
echo "🔍 Running analysis of spec files with integrated GitHub posting"
127-
128-
# Debug: Check if GITHUB_TOKEN is set
129-
if [ -n "$GITHUB_TOKEN" ]; then
130-
TOKEN_PREFIX="${GITHUB_TOKEN:0:10}"
131-
echo "✅ GITHUB_TOKEN is set (prefix: ${TOKEN_PREFIX}...)"
132-
else
133-
echo "❌ GITHUB_TOKEN is NOT set or empty - will fall back to SYSTEM_ACCESSTOKEN"
134-
fi
126+
echo "🔐 GitHub PAT will be fetched from Key Vault: mariner-pipelines-kv/cblmarghGithubPRPat"
135127
136128
cd .pipelines/prchecks/CveSpecFilePRCheck
137129
chmod +x run-pr-check.sh

‎.pipelines/prchecks/CveSpecFilePRCheck/GENERATE_BOT_PAT.md‎

Lines changed: 222 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,222 @@
1+
# Generate New GitHub PAT for CBL-Mariner-Bot
2+
3+
## Background
4+
The current GitHub Personal Access Token (PAT) for the CBL-Mariner-Bot account has expired. This token is used by:
5+
1. **Azure DevOps Pipeline** - To post initial antipattern detection comments on PRs
6+
2. **Azure Function** - To add labels and post challenge-related updates
7+
8+
## Impact of Expired Token
9+
- PR checks fail with `401 Bad credentials` errors
10+
- No automated comments on PRs for antipattern detection
11+
- RADAR system cannot post detection reports or labels
12+
13+
## Steps to Generate New PAT
14+
15+
### 1. Log into CBL-Mariner-Bot Account
16+
- Go to https://github.com/login
17+
- Sign in with CBL-Mariner-Bot credentials
18+
- **Contact:** Team admin or whoever manages the bot account credentials
19+
20+
### 2. Navigate to PAT Settings
21+
- Click on your profile picture (top-right corner)
22+
- Go to **Settings** → **Developer settings** (bottom of left sidebar)
23+
- Click **Personal access tokens** → **Tokens (classic)**
24+
- URL: https://github.com/settings/tokens
25+
26+
### 3. Generate New Token
27+
Click **"Generate new token"** → **"Generate new token (classic)"**
28+
29+
### 4. Configure Token Settings
30+
31+
**Token Name:** (Recommended)
32+
```
33+
Azure DevOps Pipeline - PR Checks & RADAR
34+
```
35+
36+
**Expiration:** (Choose one)
37+
- ✅ **Recommended:** `No expiration` (for production stability)
38+
- Alternative: `1 year` (requires annual renewal)
39+
40+
**Scopes:** (Select these checkboxes)
41+
- ✅ **repo** (Full control of private repositories)
42+
- This includes:
43+
- `repo:status` - Commit status
44+
- `repo_deployment` - Deployment status
45+
- `public_repo` - Public repositories
46+
- `repo:invite` - Repository invitations
47+
- ✅ **workflow** (Update GitHub Action workflows)
48+
49+
**Other scopes:** Leave unchecked
50+
51+
### 5. Generate and Copy Token
52+
- Scroll to bottom and click **"Generate token"**
53+
- ⚠️ **CRITICAL:** Copy the token immediately - you won't see it again!
54+
- Token format: `ghp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx` (40 characters)
55+
56+
### 6. Verify Token (Optional but Recommended)
57+
Test the token works:
58+
```bash
59+
curl -H "Authorization: token YOUR_NEW_TOKEN_HERE" https://api.github.com/user
60+
```
61+
62+
Expected output:
63+
```json
64+
{
65+
"login": "CBL-Mariner-Bot",
66+
"type": "User",
67+
...
68+
}
69+
```
70+
71+
If you see `"message": "Bad credentials"`, the token is invalid.
72+
73+
---
74+
75+
## Updating the Token in Azure
76+
77+
After generating the new token, it must be updated in **TWO** locations:
78+
79+
### Location 1: Azure Key Vault (for Azure Function)
80+
**Key Vault Name:** `mariner-pipelines-kv`
81+
**Secret Name:** `cblmarghGithubPRPat`
82+
83+
**Update via Azure CLI:**
84+
```bash
85+
az keyvault secret set \
86+
--vault-name mariner-pipelines-kv \
87+
--name cblmarghGithubPRPat \
88+
--value "ghp_YOUR_NEW_TOKEN_HERE"
89+
```
90+
91+
**Update via Azure Portal:**
92+
1. Go to https://portal.azure.com
93+
2. Search for `mariner-pipelines-kv`
94+
3. Go to **Secrets** (left sidebar)
95+
4. Click on `cblmarghGithubPRPat`
96+
5. Click **"+ New Version"**
97+
6. Paste new token value
98+
7. Click **"Create"**
99+
100+
### Location 2: Azure DevOps Pipeline Variables (for PR Check Pipeline)
101+
102+
You need to update **BOTH** of these variables (they should have the same value):
103+
- `cblmarghGithubPRPat`
104+
- `githubPrPat`
105+
106+
**Update via Azure DevOps UI:**
107+
1. Go to your Azure DevOps project
108+
2. Navigate to **Pipelines** → **Library**
109+
3. Find the variable group OR go to the specific pipeline settings
110+
4. Update both variable values with the new token
111+
5. ✅ Check "Keep this value secret" (lock icon)
112+
6. Click **Save**
113+
114+
**Alternative: Update via Pipeline YAML (Not Recommended for Secrets)**
115+
- Better to use UI to keep tokens encrypted
116+
117+
---
118+
119+
## Verification Steps
120+
121+
### 1. Verify Key Vault Update
122+
```bash
123+
az keyvault secret show \
124+
--vault-name mariner-pipelines-kv \
125+
--name cblmarghGithubPRPat \
126+
--query "value" -o tsv | head -c 10
127+
```
128+
Expected: `ghp_XXXXXX` (first 10 chars of new token)
129+
130+
### 2. Verify Azure Function
131+
- Go to Azure Portal → Function App `radarfunc`
132+
- The function will automatically pick up the new token from Key Vault
133+
- No restart needed (uses DefaultAzureCredential)
134+
135+
### 3. Verify Pipeline
136+
- Trigger a test PR check pipeline run
137+
- Check logs for: `✅ GITHUB_TOKEN is set (prefix: ghp_XXXXXX...)`
138+
- Verify the prefix matches your NEW token (not `ghp_4qL6t6...`)
139+
140+
### 4. End-to-End Test
141+
- Create a test PR with an antipattern (e.g., far-future CVE year)
142+
- Verify bot posts initial comment ✅
143+
- Verify labels are added ✅
144+
- Verify no 401 errors ❌
145+
146+
---
147+
148+
## Troubleshooting
149+
150+
### Issue: "Bad credentials" Error
151+
**Cause:** Token may not be authorized for Microsoft organization
152+
153+
**Solution:**
154+
1. Go to https://github.com/settings/tokens
155+
2. Find your new token in the list
156+
3. Click **"Configure SSO"** next to it
157+
4. Click **"Authorize"** next to `microsoft` organization
158+
5. Confirm authorization
159+
160+
### Issue: "Resource not found" Error
161+
**Cause:** Missing required scopes
162+
163+
**Solution:**
164+
- Regenerate token with `repo` and `workflow` scopes
165+
- Delete old token to avoid confusion
166+
167+
### Issue: Pipeline still uses old token
168+
**Cause:** Variable not updated in correct location
169+
170+
**Solution:**
171+
- Check BOTH pipeline variables: `cblmarghGithubPRPat` AND `githubPrPat`
172+
- Verify both have the NEW token value
173+
- Check pipeline YAML uses correct variable name (line 120)
174+
175+
---
176+
177+
## Security Best Practices
178+
179+
✅ **DO:**
180+
- Use "No expiration" for production stability
181+
- Enable SSO authorization for Microsoft org
182+
- Store in Key Vault (encrypted at rest)
183+
- Mark as secret in Azure DevOps
184+
- Document token purpose and location
185+
- Test token before deploying
186+
187+
❌ **DON'T:**
188+
- Commit token to git repository
189+
- Share token in chat/email
190+
- Use personal account token for bot operations
191+
- Store in plain text files
192+
- Reuse tokens across multiple systems
193+
194+
---
195+
196+
## Contact Information
197+
198+
**If you need help:**
199+
- Primary contact: [Your team lead or admin name]
200+
- Bot account owner: [Whoever manages CBL-Mariner-Bot credentials]
201+
- Azure subscription owner: [Person with Key Vault access]
202+
203+
**Current Status (as of October 24, 2025):**
204+
- ❌ Old token: `ghp_4qL6t6...` - EXPIRED
205+
- ⏳ New token: Waiting for generation
206+
- 🔄 Temporary workaround: Using personal token (not recommended for production)
207+
208+
---
209+
210+
## After Token Update
211+
212+
Once the new token is generated and deployed:
213+
214+
1. ✅ Test with a PR check run
215+
2. ✅ Update this document with generation date
216+
3. ✅ Set calendar reminder for renewal (if not "no expiration")
217+
4. ✅ Delete old token from GitHub settings
218+
5. ✅ Notify team that system is operational
219+
220+
**Generated by:** [Your name]
221+
**Date:** October 24, 2025
222+
**Last Updated:** October 24, 2025

‎.pipelines/prchecks/CveSpecFilePRCheck/GitHubClient.py‎

Lines changed: 78 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -19,10 +19,59 @@
1919
from typing import Dict, List, Any, Optional
2020
from AntiPatternDetector import Severity
2121

22+
# Azure Key Vault imports
23+
try:
24+
from azure.identity import DefaultAzureCredential
25+
from azure.keyvault.secrets import SecretClient
26+
KEY_VAULT_AVAILABLE = True
27+
except ImportError:
28+
KEY_VAULT_AVAILABLE = False
29+
logging.warning("Azure Key Vault SDK not available - will use environment variables only")
30+
2231
# Configure logging
2332
logging.basicConfig(level=logging.INFO)
2433
logger = logging.getLogger("github-client")
2534

35+
def fetch_github_token_from_keyvault() -> Optional[str]:
36+
"""
37+
Fetch GitHub PAT from Azure Key Vault using Managed Identity.
38+
39+
Returns:
40+
str: GitHub PAT token from Key Vault, or None if unavailable
41+
"""
42+
if not KEY_VAULT_AVAILABLE:
43+
logger.warning("⚠️ Azure Key Vault SDK not available - skipping Key Vault token fetch")
44+
return None
45+
46+
try:
47+
# Configuration from security-config-dev.json
48+
vault_name = "mariner-pipelines-kv"
49+
secret_name = "cblmarghGithubPRPat"
50+
vault_url = f"https://{vault_name}.vault.azure.net"
51+
52+
logger.info(f"🔐 Fetching GitHub PAT from Key Vault: {vault_name}/{secret_name}")
53+
54+
# Use DefaultAzureCredential (will use Managed Identity in pipeline)
55+
credential = DefaultAzureCredential()
56+
secret_client = SecretClient(vault_url=vault_url, credential=credential)
57+
58+
# Fetch the secret
59+
secret = secret_client.get_secret(secret_name)
60+
token = secret.value
61+
62+
if token and token.strip():
63+
token_prefix = token[:10] if len(token) >= 10 else token
64+
logger.info(f"✅ Successfully fetched GitHub PAT from Key Vault (prefix: {token_prefix}...)")
65+
return token
66+
else:
67+
logger.warning("⚠️ Key Vault secret is empty")
68+
return None
69+
70+
except Exception as e:
71+
logger.warning(f"⚠️ Failed to fetch token from Key Vault: {e}")
72+
logger.warning(" Will fall back to environment variables")
73+
return None
74+
2675
class CheckStatus(Enum):
2776
"""GitHub Check API status values"""
2877
SUCCESS = "success" # All good, everything passes
@@ -37,29 +86,38 @@ class GitHubClient:
3786
"""Client for interacting with GitHub API for PR checks and comments"""
3887

3988
def __init__(self):
40-
"""Initialize the GitHub client using environment variables for auth"""
41-
# Try multiple token environment variables in order of preference
42-
token_vars = [
43-
"GITHUB_TOKEN", # Prioritize CBL-Mariner bot PAT from key vault
44-
"SYSTEM_ACCESSTOKEN", # Fall back to Azure DevOps OAuth token
45-
"GITHUB_ACCESS_TOKEN",
46-
"AZDO_GITHUB_TOKEN"
47-
]
48-
89+
"""Initialize the GitHub client using Key Vault or environment variables for auth"""
4990
self.token = None
50-
for var in token_vars:
51-
token_value = os.environ.get(var, "")
52-
# Only use non-empty tokens
53-
if token_value and token_value.strip():
54-
self.token = token_value
55-
token_prefix = token_value[:10] if len(token_value) >= 10 else token_value
56-
logger.info(f"✅ Using {var} for GitHub authentication (prefix: {token_prefix}...)")
57-
break
58-
elif var in os.environ:
59-
logger.warning(f"⚠️ {var} is set but empty - skipping")
91+
92+
# FIRST: Try to fetch token from Azure Key Vault (single source of truth)
93+
logger.info("🔐 Attempting to fetch GitHub PAT from Key Vault...")
94+
kv_token = fetch_github_token_from_keyvault()
95+
if kv_token:
96+
self.token = kv_token
97+
logger.info("✅ Using GitHub PAT from Key Vault")
98+
else:
99+
# FALLBACK: Try environment variables (for local testing or when Key Vault unavailable)
100+
logger.info("⚠️ Key Vault token not available, trying environment variables...")
101+
token_vars = [
102+
"GITHUB_TOKEN", # Explicit GitHub token
103+
"SYSTEM_ACCESSTOKEN", # Azure DevOps OAuth token
104+
"GITHUB_ACCESS_TOKEN",
105+
"AZDO_GITHUB_TOKEN"
106+
]
107+
108+
for var in token_vars:
109+
token_value = os.environ.get(var, "")
110+
# Only use non-empty tokens
111+
if token_value and token_value.strip():
112+
self.token = token_value
113+
token_prefix = token_value[:10] if len(token_value) >= 10 else token_value
114+
logger.info(f"✅ Using {var} for GitHub authentication (prefix: {token_prefix}...)")
115+
break
116+
elif var in os.environ:
117+
logger.warning(f"⚠️ {var} is set but empty - skipping")
60118

61119
if not self.token:
62-
logger.error("❌ No valid GitHub token found in environment variables")
120+
logger.error("❌ No valid GitHub token found in Key Vault or environment variables")
63121

64122
# Get repository details from environment variables
65123
self.repo_name = os.environ.get("GITHUB_REPOSITORY", "") # Format: owner/repo

‎.pipelines/prchecks/CveSpecFilePRCheck/requirements.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
openai>=1.63.0
22
azure-identity>=1.15.0
33
azure-storage-blob>=12.19.0
4+
azure-keyvault-secrets>=4.7.0
45
requests>=2.25.0

0 commit comments

Comments
 (0)