Implement k8s multi cluster rollback

[ffjlabo]

What this PR does:

I implemented the rollback stage for the k8s multicluster plugin.

Spec summary

• Deploy the previous version for each deploy target at the same time.
• Not pruning to be as safe as possible. (same as k8s plugin)

RFC: https://github.com/pipe-cd/pipecd/blob/master/docs/rfcs/0014-multi-cluster-deployment-for-k8s.md

Why we need it:

We want to support the feature to deploy multi-cluster for k8s.

Which issue(s) this PR fixes:

Part of #5006

Does this PR introduce a user-facing change?:

• How are users affected by this change:
• Is this breaking change:
• How to migrate (if breaking change):

[codecov]

Codecov Report

Attention: Patch coverage is 85.71429% with 13 lines in your changes missing coverage. Please review.

Project coverage is 27.23%. Comparing base (e54cd83) to head (9a2c241).
Report is 4 commits behind head on master.

Files with missing lines	Patch %	Lines
pkg/plugin/sdk/deployment.go	0.00%	10 Missing ⚠️
...gin/kubernetes_multicluster/deployment/rollback.go	96.29%	2 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #5765      +/-   ##
==========================================
+ Coverage   27.14%   27.23%   +0.09%     
==========================================
  Files         506      506              
  Lines       53705    53788      +83     
==========================================
+ Hits        14577    14651      +74     
- Misses      38013    38022       +9     
  Partials     1115     1115              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[t-kikuc]

[ASK] What if one rollback fails and others are still rolling back?

Cancel all of them? Or still continue rolback?

[ffjlabo]

Thank you for pointing that out!
The current implementation is cancelling all.

I forgot to implement to continue rollback. will do it.

BTW, I described that in the rfc 🙏
https://github.com/pipe-cd/pipecd/blob/master/docs/rfcs/0014-multi-cluster-deployment-for-k8s.md

[ffjlabo]

Fixed 362b45e

[t-kikuc]

Thanks, i understood here:

You can select the following behavior by setting the stage.
Rollback if any app fails
Rollback if all apps fail

[t-kikuc]

[ASK] Which should we choose here? (I don't have a strong opinion)

• a. SUCCESS if at least one succeeds
• b. SUCCESS if all succeed

This is difficult... Any choice seems confusing, and I want a status like "Some Success / Some Failure" 😅

If at least one of the rollback processes succeeds, we consider the rollback successful. This ensures that the rollback is executed for other environments even if one of the deployment environments is inaccessible.

https://github.com/pipe-cd/pipecd/blob/master/docs/rfcs/0014-multi-cluster-deployment-for-k8s.md#qicksync

[ffjlabo]

We should choose a. SUCCESS if at least one succeeds to ensure rollback when one of the cluster has some trouble but want to rollback another cluster.

This is difficult... Any choice seems confusing, and I want a status like "Some Success / Some Failure" 😅

I know your point, but this spec is only for this plugin. 😇
I plan to add a log to show the each rollback status and the whole status Instead of adding status.

[Warashi]

👍🏻

[khanhtc1202]

👍