Skip to content

Conversation

@zerofltexx
Copy link
Contributor

@zerofltexx zerofltexx commented Nov 7, 2025

  • This change is worth documenting at https://docs.all-hands.dev/
  • Include this change in the Release Notes. If checked, you must provide an end-user friendly description for your change below

End-user friendly description

This PR adds enterprise/SaaS support for Azure DevOps integration. It builds on top of /pull/11243 PR by adding SSO authentication, user identity mapping, and webhook security features required for multi-tenant SaaS deployment.


Summary

This PR extends the base Azure DevOps integration PR with enterprise-specific features.

What This PR Adds

  1. SSO Integration
  • Keycloak identity provider configuration
  • Automatic background enrichment on user login
Screenshot 2025-11-07 165250
  1. User Identity Resolution
  • Resolves Azure AD Object ID → Azure DevOps User ID via Graph API
  • Stores mapping in Keycloak user attributes (azure_devops_id)
  • Why needed: Webhook payloads contain Azure DevOps User IDs (VSID), not Azure AD IDs. Without this mapping, we can't identify which OpenHands user triggered the webhook.
  1. Webhook Security
  • Webhook verification using (organization, subscriptionId, api_key) lookup
  • Deduplication using Redis
  1. Enterprise Service Implementation
  • Extends base AzureDevOpsServiceImpl with SaaS-specific features
  • Adds dual authentication support (OAuth JWT + PAT auto-detection)
  • Token manager integration for multi-user environments
  1. Webhook Event Processing
  • Processes work item and PR comment webhooks
  • AI solvability analysis for work items
  • Acknowledgment messages posted back to Azure DevOps
  • User identity resolution from webhook payloads
Screenshot 2025-11-07 165156 Screenshot 2025-11-07 164824

Prerequisite
Please follow the README.md in All-Hands-AI/OpenHands-Cloud/pull/238

Azure DevOps (Client side - unfortunately this can't be automated based on my research)

Screenshot 2025-11-07 165526
  • Create an API key. Have this ready for the next step

  • Create the webhook, since it is at project level, this pair needs to be created in every project. In HTTP headers paste the API key like so:

Screenshot 2025-11-07 165842 image

To-do

  • Test webhook API key method
  • OpenHands-Cloud ADO PR
  • Test auto discover Organization from SSO

Related Issues

@zerofltexx zerofltexx force-pushed the feat/azure-devops-support-enterprise branch 25 times, most recently from 17058e1 to 14b9c54 Compare November 10, 2025 03:06
@zerofltexx zerofltexx marked this pull request as ready for review November 10, 2025 03:52
@zerofltexx zerofltexx requested a review from amanape as a code owner November 10, 2025 03:52
@mamoodi mamoodi requested review from malhotra5 and neubig November 10, 2025 14:21
@zerofltexx zerofltexx force-pushed the feat/azure-devops-support-enterprise branch 6 times, most recently from 95fd5c4 to e90b397 Compare November 21, 2025 10:43
Copy link
Contributor

@neubig neubig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm happy to take a look at this when merge conflicts are resolved and tests are passing, please re-request review then!

zerofltexx pushed a commit to zerofltexx/OpenHands that referenced this pull request Nov 22, 2025
Resolved merge conflicts between enterprise Azure DevOps integration (PR OpenHands#11667)
and the base Azure DevOps integration that was merged into main (PR OpenHands#11243).

Key conflict resolutions:
- Kept X-VSS-UserData header parsing for subject ID extraction (needed for
  work item assignment matching in enterprise webhooks)
- Kept Markdown to HTML conversion for Azure DevOps comment formatting
- Kept get_work_item method (used by enterprise webhook code)
- Kept selected_branch handling for linked development branches
- Kept users/openhands/{id} branch naming format
- Added add_work_item_reaction method from main branch

All enterprise-specific features have been preserved while incorporating
updates from the main branch.

Co-authored-by: openhands <[email protected]>
zerofltexx pushed a commit to zerofltexx/OpenHands that referenced this pull request Nov 22, 2025
Resolved merge conflicts between enterprise Azure DevOps integration (PR OpenHands#11667)
and the base Azure DevOps integration that was merged into main (PR OpenHands#11243).

Key conflict resolutions:
- Kept X-VSS-UserData header parsing for subject ID extraction (needed for
  work item assignment matching in enterprise webhooks)
- Kept Markdown to HTML conversion for Azure DevOps comment formatting
- Kept get_work_item method (used by enterprise webhook code)
- Kept selected_branch handling for linked development branches
- Kept users/openhands/{id} branch naming format
- Added add_work_item_reaction method from main branch

All enterprise-specific features have been preserved while incorporating
updates from the main branch.

Co-authored-by: openhands <[email protected]>
@zerofltexx zerofltexx force-pushed the feat/azure-devops-support-enterprise branch 3 times, most recently from f6bcc5b to acfb349 Compare November 22, 2025 19:47
@zerofltexx zerofltexx requested a review from neubig November 22, 2025 19:55
@zerofltexx zerofltexx force-pushed the feat/azure-devops-support-enterprise branch from 87c19b8 to 9c09ccd Compare November 30, 2025 06:33
@zerofltexx zerofltexx force-pushed the feat/azure-devops-support-enterprise branch 3 times, most recently from 7647ce1 to 38170e5 Compare November 30, 2025 06:40
@zerofltexx
Copy link
Contributor Author

Fix: Mainly to rename Azure DevOps webhook table migration (083 -> 084). New migration file took over 83 & merged to main before this one.

  • Clean up unused columns from webhook model
  • Extract duplicate user lookup logic to _resolve_keycloak_user_id()
  • Standardize logging prefixes to [Azure DevOps]
  • Return 400 status codes for webhook errors (matching GitLab pattern)
  • Add module-level TokenManager instance
  • Fix auto-login for Azure DevOps users
  • Remove duplicate import in auth.py

The failed test is due to this
image

@zerofltexx zerofltexx force-pushed the feat/azure-devops-support-enterprise branch 8 times, most recently from 4f13bfd to 1f25ce7 Compare December 5, 2025 10:09
This commit adds enterprise-grade Azure DevOps support with webhook-based automation
and service principal authentication for multi-tenant SaaS deployments.

Key changes:
- Implemented webhook event processing for work items and pull requests
- Added service principal authentication for automated operations
- Implemented Azure DevOps manager for handling webhook events
- Added work item and PR comment support with @mention detection
- Created callback processor for asynchronous webhook responses
- Added webhook registration and management endpoints
- Implemented Azure DevOps user enrichment and token management
- Added database schema for webhook storage
- Fixed API response format handling for comments and field values
- Removed linter suppressions for cleaner code

The implementation enables:
- Automated responses to work item assignments and @mentions
- Pull request comment handling with inline code review support
- Service principal-based API operations for scalability
- Multi-organization support with webhook deduplication
- Integration with Keycloak for user management
- Webhook-driven conversation creation and management
@zerofltexx zerofltexx force-pushed the feat/azure-devops-support-enterprise branch from 1f25ce7 to bf7aaa8 Compare December 5, 2025 14:48
@neubig
Copy link
Contributor

neubig commented Dec 6, 2025

Hi, just a note, I'll circle back to reviewing this next week after we (hopefully) finish with shipping OpenHands v1 to the OpenHands cloud.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants