Test-Automation for rcm installation

[voigt]

We want to automate test runs to ensure the reliable installation of RCM and shims.

What to test

• test installation of RCM Helm Chart
• test successful installation of common shims (to be defined; I propose to start with the spin shim and expand from there)

Where to test

As RCM is messing with nodes' filesystem, using it on as many Kubernetes flavors as possible is crucial. Therefore, a minimum of covered Kubernetes distributions would be:

• kind
• k3s
• k3d

Ideally, we'd like to test the managed services of major cloud providers:

• AWS EKS
• Azure AKS
• GCP GKE
• ... potentially others like Civo, Digital Ocean, etc.

However, the main issue will be the associated cost of creating clusters during CI runtime. Maybe we'll find a way to get some sponsoring?

[phyrog]

Some pointers to useful GH actions:

• https://github.com/marketplace/actions/absaoss-k3d-action
• https://github.com/marketplace/actions/kind-cluster
• https://github.com/marketplace/actions/helm-chart-testing

[voigt]

security-profiles-operator faces similar challenges as rcm. Here their GH workflow to test within kind: https://github.com/kubernetes-sigs/security-profiles-operator/blob/main/.github/workflows/olm_tests.yaml

[phyrog]

Can we close this issue with the merge of #234?

[vdice]

I'd vote we either close this one and track next test automation goals (other distros, other shims, more scenarios) in a new ticket or continue tracking in this ticket but remove from the first milestone (I think #234 satisfies test automation needs for the first mvp release).