Skip to content

Latest commit

 

History

History
29 lines (23 loc) · 1.19 KB

README.md

File metadata and controls

29 lines (23 loc) · 1.19 KB

Workaround ETCD Snapshots Part Files Issue

To workaround issue gh-30662 please select one of the following deployment options.

Option A - cleanup file temp files

This script runs on each etcd node in a while true loop every 5 minutes looking for leftover part files. If it finds part files older than 15 minutes, it will delete them. This is to prevent deleting a part file that is currently in-use.

Changes to restore process

None, the restore process is unchanged.

Installation

kubectl apply -f delete-part-files.yaml

Option B - alternative s3 snapshots

This script replaces the recurring snapshot functionality in RKE with a Kubernetes job that runs every 12 hours.

Changes to restore process

  • You will need to manually take a new snapshot
  • Download the snapshot from S3 on all etcd nodes
  • Rename the old snapshot to the new snapshot filename
  • Restore the S3 snapshot in Rancher UI by selecting the new snapshot name

Installation

  • Disable recurring snapshots in Rancher/RKE
  • At a minimum, alt-s3-sync.yaml must be modified (remember to base64 the values) to reflect the s3 details
kubectl apply -f alt-s3-sync.yaml