Skip to content

Commit 776d9b0

Browse files
marek-veberclaude
andcommitted
Address review feedback for ARO HCP proposal
Updates based on willie-yao's review comments: - Update metadata: add reviewers, fix dates, change status to implementable - Fix API versions: v1beta2 → v1beta1 throughout examples - Add "Alternatives Considered" section explaining why separate CRDs are needed - Add "Maintenance and Ownership" section clarifying ARO team ownership - Document ASO migration plan with dependency chain - Add references to ARO-HCP repository and API specifications - Note API versions: 2024-06-10-preview (private) vs 2025-12-23-preview (public) - Add ASO issue reference for key management gap - Update graduation criteria with ownership and migration commitments Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
1 parent 3949aeb commit 776d9b0

1 file changed

Lines changed: 60 additions & 11 deletions

File tree

docs/proposals/20250425-aro-hcp.md

Lines changed: 60 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,12 @@ authors:
44
- "@serngawy"
55
- "@marek-veber"
66
reviewers:
7-
- ""
8-
creation-date: 2025-04-23
9-
last-updated: 2026-02-20
10-
status: implemented
7+
- "@willie-yao"
8+
- "@jackfrancis"
9+
- "@bryan-cox"
10+
creation-date: 2025-04-25
11+
last-updated: 2026-03-11
12+
status: implementable
1113
---
1214

1315

@@ -23,6 +25,8 @@ status: implemented
2325
- [Proposal](#proposal)
2426
- [User Stories](#user-stories)
2527
- [Implementation Details](#implementation-details)
28+
- [Alternatives Considered](#alternatives-considered)
29+
- [Maintenance and Ownership](#maintenance-and-ownership)
2630
- [Risks and Mitigations](#risks-and-mitigations)
2731
- [Graduation Criteria](#graduation-criteria)
2832
- [Implementation History](#implementation-history)
@@ -33,6 +37,8 @@ status: implemented
3337

3438
The Cluster API Provider for Azure (CAPZ) extends Kubernetes Cluster API functionality to Microsoft Azure environments, facilitating the management and lifecycle of Kubernetes clusters on Azure. This proposal outlines the integration of Azure Red Hat OpenShift (ARO) Hosted Control Plane (HCP) clusters within CAPZ using Azure Service Operator (ASO) for resource provisioning.
3539

40+
ARO HCP is an evolution of Azure Red Hat OpenShift that uses a hosted control plane architecture. For more information about ARO HCP, see the [ARO-HCP repository](https://github.com/Azure/ARO-HCP).
41+
3642
The implementation leverages ASO resources embedded directly in Custom Resource specifications, providing a declarative, Kubernetes-native approach to managing ARO HCP infrastructure.
3743

3844

@@ -115,9 +121,13 @@ type AROControlPlaneSpec struct {
115121
// - HcpOpenShiftCluster resource
116122
// - HcpOpenShiftClustersExternalAuth resource (if external auth is needed)
117123
//
118-
// Example resources:
124+
// Example resources (using private preview API v1api20240610preview):
119125
// - redhatopenshift.azure.com/v1api20240610preview/HcpOpenShiftCluster
120126
// - redhatopenshift.azure.com/v1api20240610preview/HcpOpenShiftClustersExternalAuth
127+
//
128+
// Note: v1api20240610preview is the private preview API. Migration to public preview
129+
// API (v1api20251223preview) is planned. See API specs at:
130+
// https://github.com/Azure/ARO-HCP/tree/main/api/redhatopenshift/resource-manager/Microsoft.RedHatOpenShift/hcpclusters/preview
121131
Resources []runtime.RawExtension `json:"resources,omitempty"`
122132

123133
// IdentityRef is a reference to an identity to be used when reconciling the ARO control plane.
@@ -203,7 +213,7 @@ type ResourceStatus struct {
203213
An example of AROControlPlane CR using resources mode:
204214

205215
```yaml
206-
apiVersion: controlplane.cluster.x-k8s.io/v1beta2
216+
apiVersion: controlplane.cluster.x-k8s.io/v1beta1
207217
kind: AROControlPlane
208218
metadata:
209219
name: my-cluster
@@ -343,7 +353,7 @@ status:
343353
This example shows AROControlPlane using ASO credentials instead of identityRef:
344354
345355
```yaml
346-
apiVersion: controlplane.cluster.x-k8s.io/v1beta2
356+
apiVersion: controlplane.cluster.x-k8s.io/v1beta1
347357
kind: AROControlPlane
348358
metadata:
349359
name: hcp-cluster
@@ -471,7 +481,7 @@ type AROMachinePoolSpec struct {
471481
// Resources are embedded ASO resources to be managed by this AROMachinePool.
472482
// Must include HcpOpenShiftClustersNodePool resource.
473483
//
474-
// Example:
484+
// Example (using private preview API v1api20240610preview):
475485
// - redhatopenshift.azure.com/v1api20240610preview/HcpOpenShiftClustersNodePool
476486
Resources []runtime.RawExtension `json:"resources"`
477487
}
@@ -498,7 +508,7 @@ type AROMachinePoolStatus struct {
498508
An example of AROMachinePool CR:
499509

500510
```yaml
501-
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
511+
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
502512
kind: AROMachinePool
503513
metadata:
504514
name: my-cluster-mp1
@@ -620,7 +630,7 @@ type AROClusterInitializationStatus struct {
620630
An example of AROCluster CR:
621631

622632
```yaml
623-
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
633+
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
624634
kind: AROCluster
625635
metadata:
626636
name: my-cluster
@@ -1066,7 +1076,7 @@ spec.properties.etcd.dataEncryption.customerManaged.kms.activeKey.version
10661076
**Solution**: Add the Vault resource to **AROCluster.spec.resources[]** (not AROControlPlane):
10671077
10681078
```yaml
1069-
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
1079+
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
10701080
kind: AROCluster
10711081
metadata:
10721082
name: my-cluster
@@ -1148,6 +1158,8 @@ spec:
11481158
11491159
- **Azure SDK**: Used only for encryption key management (when identityRef is set)
11501160
- KeyVault keys (create/retrieve key versions within existing vault)
1161+
- **Note**: This is a temporary gap in ASO coverage. ASO supports Vault CRDs but not key management within vaults. Upstream tracking: [Azure/azure-service-operator#3188](https://github.com/Azure/azure-service-operator/issues/3188)
1162+
- **Future**: Once ASO adds support for key version retrieval, the keyvaults SDK service can be removed entirely
11511163
11521164
### Validation and Testing
11531165
@@ -1193,6 +1205,40 @@ Resource Deployment
11931205
- Testing of error conditions and recovery
11941206
- Testing of both authentication modes (CAPZ managed vs ASO credential-based)
11951207
1208+
## Alternatives Considered
1209+
1210+
### Using AzureASOManagedControlPlane
1211+
1212+
AzureASOManagedControlPlane is designed for AKS managed clusters using `containerservice.azure.com` resources. ARO HCP uses fundamentally different Azure resource types (`redhatopenshift.azure.com/HcpOpenShiftCluster`) with ARO-specific requirements including ETCD encryption with Key Vault integration, external authentication resources, and different node pool APIs. The architectural differences are significant enough that separate CRDs provide clearer API boundaries and simpler controller logic than trying to unify both platforms under a single type.
1213+
1214+
### Pure ASO Without CAPZ Controllers
1215+
1216+
While ASO provides Azure resource management, CAPZ controllers add Cluster API integration (lifecycle management, dependency orchestration, status aggregation) and automation features like encryption key version injection. Users who don't need CAPI integration can use ASO directly with `identityRef: nil` mode.
1217+
1218+
## Maintenance and Ownership
1219+
1220+
The ARO HCP integration is **owned and maintained by the Azure Red Hat OpenShift (ARO) team at Red Hat**. This includes the AROControlPlane, AROMachinePool, and AROCluster CRDs, controllers, webhooks, and ARO-specific documentation.
1221+
1222+
**CAPZ core team responsibilities** are limited to maintaining shared infrastructure (ASO integration patterns, common libraries) and reviewing ARO pull requests for CAPZ architectural alignment. The CAPZ core team is **not expected to own or maintain ARO-specific logic**.
1223+
1224+
### Azure SDK Usage and ASO Migration Plan
1225+
1226+
The keyvaults service currently uses Azure SDK for key version retrieval because ASO doesn't yet support Key Vault key management CRDs (only Vault resources). This SDK usage is temporary and will be removed once ASO coverage is available.
1227+
1228+
**ARO HCP API Versions**:
1229+
- **2024-06-10-preview**: Private preview API version (currently used in this implementation)
1230+
- **2025-12-23-preview**: Public preview API version (planned migration target)
1231+
1232+
API specifications are maintained in the [ARO-HCP repository](https://github.com/Azure/ARO-HCP/tree/main/api/redhatopenshift/resource-manager/Microsoft.RedHatOpenShift/hcpclusters/preview).
1233+
1234+
**Dependency chain for full ASO migration**:
1235+
1. ARO HCP public preview completion (pending Microsoft release formalities)
1236+
2. ARO HCP API specification merged into Azure REST API specs repository
1237+
3. ASO generation of ARO HCP CRDs from updated specs
1238+
4. Backporting ASO changes to the release version used by CAPZ
1239+
1240+
Once these dependencies are resolved and ASO supports key management, the keyvaults SDK service will be removed entirely. The ARO team commits to migrating to pure ASO-native implementation at that time.
1241+
11961242
## Risks and Mitigations
11971243
11981244
- **ASO API Changes**: Monitor ASO releases and update resource definitions accordingly
@@ -1209,6 +1255,9 @@ Resource Deployment
12091255
- ✅ Encryption key management working correctly
12101256
- ✅ ExternalAuth configuration functioning properly
12111257
- ✅ Clean code with field-based mode removed (~5,400 lines removed)
1258+
- ✅ Maintenance ownership documented and agreed upon
1259+
- ✅ Plan documented to remove keyvaults SDK service when ASO supports key version retrieval
1260+
- ✅ Acknowledgment that CAPZ's long-term direction is toward pure ASO-native controllers
12121261
12131262
## Implementation History
12141263

0 commit comments

Comments
 (0)