[WIP] KMS: add a post check function to revision controller #2121
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,116 @@ | ||||||||||||||||
| package kms | ||||||||||||||||
|
|
||||||||||||||||
| import ( | ||||||||||||||||
| "context" | ||||||||||||||||
| "encoding/json" | ||||||||||||||||
| "fmt" | ||||||||||||||||
| "slices" | ||||||||||||||||
|
|
||||||||||||||||
| "github.com/openshift/api/features" | ||||||||||||||||
| "github.com/openshift/cluster-kube-apiserver-operator/pkg/operator/operatorclient" | ||||||||||||||||
| "github.com/openshift/library-go/pkg/operator/configobserver/featuregates" | ||||||||||||||||
| "github.com/openshift/library-go/pkg/operator/revisioncontroller" | ||||||||||||||||
| corev1 "k8s.io/api/core/v1" | ||||||||||||||||
| apierrors "k8s.io/apimachinery/pkg/api/errors" | ||||||||||||||||
| metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" | ||||||||||||||||
| "k8s.io/apimachinery/pkg/runtime" | ||||||||||||||||
| "k8s.io/apimachinery/pkg/runtime/serializer" | ||||||||||||||||
| utilruntime "k8s.io/apimachinery/pkg/util/runtime" | ||||||||||||||||
| apiserverv1 "k8s.io/apiserver/pkg/apis/apiserver/v1" | ||||||||||||||||
| "k8s.io/client-go/kubernetes" | ||||||||||||||||
| "k8s.io/klog/v2" | ||||||||||||||||
| ) | ||||||||||||||||
|
|
||||||||||||||||
| var ( | ||||||||||||||||
| apiserverScheme = runtime.NewScheme() | ||||||||||||||||
| apiserverCodecs = serializer.NewCodecFactory(apiserverScheme) | ||||||||||||||||
| ) | ||||||||||||||||
|
|
||||||||||||||||
| func init() { | ||||||||||||||||
| utilruntime.Must(apiserverv1.AddToScheme(apiserverScheme)) | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| func KMSRevisionPostCheck(kubeClient kubernetes.Interface, featureGateAccessor featuregates.FeatureGateAccess) revisioncontroller.RevisionPostCheckFunc { | ||||||||||||||||
| return func(ctx context.Context, revision int32) error { | ||||||||||||||||
| return validateKMSRevision(ctx, kubeClient, featureGateAccessor, revision) | ||||||||||||||||
| } | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| func validateKMSRevision(ctx context.Context, kubeClient kubernetes.Interface, featureGateAccessor featuregates.FeatureGateAccess, revision int32) error { | ||||||||||||||||
| if !featureGateAccessor.AreInitialFeatureGatesObserved() { | ||||||||||||||||
| return nil | ||||||||||||||||
| } | ||||||||||||||||
| featureGates, err := featureGateAccessor.CurrentFeatureGates() | ||||||||||||||||
| if err != nil { | ||||||||||||||||
| return nil | ||||||||||||||||
|
Comment on lines
+43
to
+45
There was a problem hiding this comment. Do not swallow feature-gate retrieval errors. At Line 45, 🔧 Proposed fix featureGates, err := featureGateAccessor.CurrentFeatureGates()
if err != nil {
- return nil
+ return fmt.Errorf("kms revision post-check: failed to read current feature gates: %w", err)
}📝 Committable suggestion
Suggested change
🧰 Tools🪛 golangci-lint (2.11.4)[error] 45-45: error is not nil (line 43) but it returns nil (nilerr) 🤖 Prompt for AI Agents |
||||||||||||||||
| } | ||||||||||||||||
| if !featureGates.Enabled(features.FeatureGateKMSEncryption) { | ||||||||||||||||
| return nil | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| revSuffix := fmt.Sprintf("-%d", revision) | ||||||||||||||||
|
|
||||||||||||||||
| encryptionSecret, err := kubeClient.CoreV1().Secrets(operatorclient.TargetNamespace).Get(ctx, "encryption-config"+revSuffix, metav1.GetOptions{}) | ||||||||||||||||
| if apierrors.IsNotFound(err) { | ||||||||||||||||
| return nil | ||||||||||||||||
|
Member
There was a problem hiding this comment. I assume that notfound revisioned Secret is totally benign?
Member
Author
There was a problem hiding this comment. I assume so, that's why I return nil here. Should I handle it differently? |
||||||||||||||||
| } | ||||||||||||||||
| if err != nil { | ||||||||||||||||
| return fmt.Errorf("kms revision post-check: failed to get encryption-config for revision %d: %w", revision, err) | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| podCM, err := kubeClient.CoreV1().ConfigMaps(operatorclient.TargetNamespace).Get(ctx, "kube-apiserver-pod"+revSuffix, metav1.GetOptions{}) | ||||||||||||||||
| if apierrors.IsNotFound(err) { | ||||||||||||||||
| return nil | ||||||||||||||||
| } | ||||||||||||||||
|
Comment on lines
+53
to
+64
There was a problem hiding this comment. Current Lines 55 and 63 return early, which skips mismatch detection entirely. For 🔧 Proposed fix- encryptionSecret, err := kubeClient.CoreV1().Secrets(operatorclient.TargetNamespace).Get(ctx, "encryption-config"+revSuffix, metav1.GetOptions{})
- if apierrors.IsNotFound(err) {
- return nil
- }
- if err != nil {
- return fmt.Errorf("kms revision post-check: failed to get encryption-config for revision %d: %w", revision, err)
- }
+ encryptionSecret, err := kubeClient.CoreV1().Secrets(operatorclient.TargetNamespace).Get(ctx, "encryption-config"+revSuffix, metav1.GetOptions{})
+ kmsConfigured := false
+ if apierrors.IsNotFound(err) {
+ kmsConfigured = false
+ } else if err != nil {
+ return fmt.Errorf("kms revision post-check: failed to get encryption-config for revision %d: %w", revision, err)
+ } else {
+ kmsConfigured = hasKMSEncryptionProvider(encryptionSecret)
+ }
podCM, err := kubeClient.CoreV1().ConfigMaps(operatorclient.TargetNamespace).Get(ctx, "kube-apiserver-pod"+revSuffix, metav1.GetOptions{})
- if apierrors.IsNotFound(err) {
- return nil
- }
if err != nil {
return fmt.Errorf("kms revision post-check: failed to get kube-apiserver-pod for revision %d: %w", revision, err)
}
-
- kmsConfigured := hasKMSEncryptionProvider(encryptionSecret)🤖 Prompt for AI Agents |
||||||||||||||||
| if err != nil { | ||||||||||||||||
| return fmt.Errorf("kms revision post-check: failed to get kube-apiserver-pod for revision %d: %w", revision, err) | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| kmsConfigured := hasKMSEncryptionProvider(encryptionSecret) | ||||||||||||||||
| // FIXME(bertinatto): plugin name is hardcoded here | ||||||||||||||||
| kmsInPod := podHasKMSSidecar(podCM, "kms-plugin") | ||||||||||||||||
|
|
||||||||||||||||
| // TODO: validation needs to cover more than this | ||||||||||||||||
| if kmsConfigured != kmsInPod { | ||||||||||||||||
| return fmt.Errorf("kms revision post-check: revision %d has mismatched KMS state: kmsConfigured=%v kmsInPod=%v", revision, kmsConfigured, kmsInPod) | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| klog.V(4).Infof("kms revision post-check passed for revision %d: kmsConfigured=%v kmsInPod=%v", revision, kmsConfigured, kmsInPod) | ||||||||||||||||
| return nil | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| func podHasKMSSidecar(podCM *corev1.ConfigMap, sidecarName string) bool { | ||||||||||||||||
| podYAML, ok := podCM.Data["pod.yaml"] | ||||||||||||||||
| if !ok { | ||||||||||||||||
| return false | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| var pod corev1.Pod | ||||||||||||||||
| if err := json.Unmarshal([]byte(podYAML), &pod); err != nil { | ||||||||||||||||
| return false | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| return slices.ContainsFunc(pod.Spec.Containers, func(c corev1.Container) bool { | ||||||||||||||||
| return c.Name == sidecarName | ||||||||||||||||
| }) | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| func hasKMSEncryptionProvider(encryptionConfig *corev1.Secret) bool { | ||||||||||||||||
| encryptionConfigBytes, ok := encryptionConfig.Data["encryption-config"] | ||||||||||||||||
| if !ok { | ||||||||||||||||
| return false | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| gvk := apiserverv1.SchemeGroupVersion.WithKind("EncryptionConfiguration") | ||||||||||||||||
| obj, _, err := apiserverCodecs.UniversalDeserializer().Decode(encryptionConfigBytes, &gvk, nil) | ||||||||||||||||
| if err != nil { | ||||||||||||||||
| return false | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| config := obj.(*apiserverv1.EncryptionConfiguration) | ||||||||||||||||
| return slices.ContainsFunc(config.Resources, func(resource apiserverv1.ResourceConfiguration) bool { | ||||||||||||||||
| return slices.ContainsFunc(resource.Providers, func(provider apiserverv1.ProviderConfiguration) bool { | ||||||||||||||||
| return provider.KMS != nil | ||||||||||||||||
| }) | ||||||||||||||||
| }) | ||||||||||||||||
| } | ||||||||||||||||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧩 Analysis chain
🏁 Script executed:
Repository: openshift/cluster-kube-apiserver-operator
Length of output: 407
Remove the personal-fork
replacedirective before merge.Line 140 reroutes all
github.com/openshift/library-goimports to a personal fork on a non-upstream feature branch (kms-plugin-lifecycle-postcondition), creating a supply-chain and reproducibility risk for CI and releases. Either remove this directive or resolve it by contributing the change upstream and using the official version.🔧 Proposed fix
-replace github.com/openshift/library-go => github.com/bertinatto/library-go v0.0.0-20260423204751-1ae67025ddad📝 Committable suggestion
🤖 Prompt for AI Agents