✨ allow preserving addons on delete

[alam0rt]

What type of PR is this?

feature

What this PR does / why we need it:

We are looking at migrating away from the managed addons (specifically kube-proxy) and wanting to use our own self-managed.

The issue is that both kube-system/daemonset/kube-proxys can't exist at the same time for obvious reason.

Removing the addon from .spec.addons will of course, delete the addon.

Deploying over the addon works, but again, removing the addon will remove the now self-managed addon.

The only way (without juggling multiple daemonsets, using taints/tolerations to run both at the same time, etc) for a clean migration is to (please let me know if a good way comes to mind):

• Deploy over the managed addon
• Remove the addon while preserving the resources on the cluster

We could potentially stop CAPA reconciliation and remove the addon manually, but kind of want to avoid that.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Special notes for your reviewer:

Checklist:

• squashed commits
• includes documentation
• includes emoji in title
• adds unit tests
• adds or updates e2e tests

Release note:

Allow preserving EKS addons.

[k8s-ci-robot]

Hi @alam0rt. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[alam0rt]

/kind feature

[alam0rt]

Speaking with my team, would it be better if the preserve field was in sigs.k8s.io/cluster-api-provider-aws/v2/controlplane/eks/api/v1beta1.Addon?

[alam0rt]

Speaking with my team, would it be better if the preserve field was in sigs.k8s.io/cluster-api-provider-aws/v2/controlplane/eks/api/v1beta1.Addon?

I went and did it. Seems cleaner.

[richardcase]

/ok-to-test

[richardcase]

@alam0rt - thanks for this. Just so i understand this correct. If you want to delete and use your own version of the addon, is this the process:

• Define the kube proxy addon with preserveOnDelete=true in AWSManagedControlPlane
• Delete the addon from AWSManagedControlPlane

[richardcase]

Also, guessing this is only for the use case where bootstrapSelfManagedAddons wasn't specified at cluster creation time?

[alam0rt]

@alam0rt - thanks for this. Just so i understand this correct. If you want to delete and use your own version of the addon, is this the process:

• Define the kube proxy addon with preserveOnDelete=true in AWSManagedControlPlane

• Delete the addon from AWSManagedControlPlane

That's correct!

[richardcase]

/test ?

[k8s-ci-robot]

@richardcase: The following commands are available to trigger required jobs:

/test pull-cluster-api-provider-aws-build

/test pull-cluster-api-provider-aws-build-docker

/test pull-cluster-api-provider-aws-test

/test pull-cluster-api-provider-aws-verify

The following commands are available to trigger optional jobs:

/test pull-cluster-api-provider-aws-apidiff-main

/test pull-cluster-api-provider-aws-e2e

/test pull-cluster-api-provider-aws-e2e-blocking

/test pull-cluster-api-provider-aws-e2e-clusterclass

/test pull-cluster-api-provider-aws-e2e-conformance

/test pull-cluster-api-provider-aws-e2e-conformance-with-ci-artifacts

/test pull-cluster-api-provider-aws-e2e-eks

/test pull-cluster-api-provider-aws-e2e-eks-gc

/test pull-cluster-api-provider-aws-e2e-eks-testing

Use /test all to run the following jobs that were automatically triggered:

pull-cluster-api-provider-aws-apidiff-main

pull-cluster-api-provider-aws-build

pull-cluster-api-provider-aws-build-docker

pull-cluster-api-provider-aws-test

pull-cluster-api-provider-aws-verify

In response to this:

/test ?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[richardcase]

/test pull-cluster-api-provider-aws-e2e-eks

[richardcase]

/approve

[k8s-ci-robot]

@alam0rt: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-cluster-api-provider-aws-e2e-blocking	b33add2	link	true	/test pull-cluster-api-provider-aws-e2e-blocking

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.