K8SPG-586: Add delete-backups finalizer

[inelpandzic]

CHANGE DESCRIPTION

Problem:
In our other operators, we have a delete-backup finalizer under backup CR, meaning we can delete the individual backup.
In PG we can’t do that, the only thing we can delete is all the backups from all the repos or backups for a particular repo.

Solution:
Add a percona.com/delete-backups finalizer that will ensure that all backups are removed when the cluster is deleted.

When the finalizer is applied, the operator will basically issue pgbackrest --stanza=db stanza-delete command that will essentially delete all the backups from all the configured repos. Besides removing all the physical backup files, finalizer will also delete all pg-backup objects since there is no reason to leave them dangling around without physical backups that correspond to them.

This is not recommended in production.

CHECKLIST

Jira

• Is the Jira ticket created and referenced properly?
• Does the Jira ticket have the proper statuses for documentation (Needs Doc) and QA (Needs QA)?
• Does the Jira ticket link to the proper milestone (Fix Version field)?

Tests

• Is an E2E test/test case added for the new feature/change?
• Are unit tests added where appropriate?

Config/Logging/Testability

• Are all needed new/changed options added to default YAML files?
• Are all needed new/changed options added to the Helm Chart?
• Did we add proper logging messages for operator actions?
• Did we ensure compatibility with the previous version or cluster upgrade process?
• Does the change support oldest and newest supported PG version?
• Does the change support oldest and newest supported Kubernetes version?

[nmarukovich]

LGFM
But let's use GetPrimaryPod method (as we discussed) when I merge custom extension deletion.

[egegunes]

should we run the second command if first one fails?

[inelpandzic]

bbe958a

[JNKPercona]

Test name	Status
custom-extensions	passed
custom-tls	passed
demand-backup	passed
finalizers	passed
init-deploy	passed
monitoring	passed
one-pod	passed
operator-self-healing	passed
pitr	passed
scaling	passed
scheduled-backup	passed
self-healing	passed
sidecars	passed
start-from-backup	passed
tablespaces	passed
telemetry-transfer	passed
upgrade-consistency	passed
upgrade-minor	passed
users	passed
We run 19 out of 19

commit: 6f323f5
image: perconalab/percona-postgresql-operator:PR-989-6f323f5ad