Skip to content

Conversation

@gnolong
Copy link
Contributor

@gnolong gnolong commented Dec 23, 2024

fix #8708
xtrabackup incremental backup for mysql ref apecloud/kubeblocks-addons#1358

Incremental Backup Support

Backup

whiteboard_exported_image (1)

1. Backup API

Incremental backups depend on existing full or incremental backups, forming a backup chain.

  • We use status.parentBackupName to indicate the parent backup of the incremental backup, which may be either an incremental or a full backup.
  • When spec.parentBackupName is empty, the backup controller selects the most recent incremental or full backup as status.parentBackupName, prioritizing scheduled backups if the backup is scheduled.
  • status.BaseBackupName indicates the full backup that the incremental backup is based on.
apiVersion: dataprotection.kubeblocks.io/v1alpha1
kind: Backup
  name: inc-backup-2
  namespace: default
spec:
  backupMethod: xtrabackup-inc
  backupPolicyName: mysql-backup-policy
  deletionPolicy: Delete
  retentionPeriod: 7d
  parentBackupName: inc-backup-1
status:
  parentBackupName: inc-backup-1
  BaseBackupName: full-backup

2. Backup Method

We introduced compatibleMethod to indicate which full backup methods are compatible with the incremental backup methods. For example:

backupMethods:
  - name: mysql-inc
    actionSetName: xtrabackup-inc
    snapshotVolumes: false
    compatibleMethod: mysql-full
  - name: mysql-full
    actionSetName: xtrabackup-full
    snapshotVolumes: false

3. Backup Deletion

When a parent backup or base backup is deleted, all backups related to them will also be deleted.

Restore

  • The restore process for incremental backups is similar to that of full backups. The backup chain must be passed to the restore workload through environment variables.
  • For point-in-time recovery, the most recent full or incremental backup closest to the specified time will be selected as the base backup.

@github-actions github-actions bot added the size/XL Denotes a PR that changes 500-999 lines. label Dec 23, 2024
@github-actions github-actions bot added size/XXL Denotes a PR that changes 1000+ lines. and removed size/XL Denotes a PR that changes 500-999 lines. labels Dec 25, 2024
@codecov
Copy link

codecov bot commented Dec 25, 2024

Codecov Report

Attention: Patch coverage is 36.49289% with 268 lines in your changes missing coverage. Please review.

Project coverage is 61.39%. Comparing base (2f60dc9) to head (bfda5cc).
Report is 32 commits behind head on main.

Files with missing lines Patch % Lines
pkg/dataprotection/restore/utils.go 0.00% 75 Missing ⚠️
pkg/dataprotection/restore/manager.go 0.00% 64 Missing ⚠️
controllers/dataprotection/utils.go 59.50% 34 Missing and 15 partials ⚠️
controllers/dataprotection/backup_controller.go 72.72% 17 Missing and 7 partials ⚠️
pkg/dataprotection/utils/utils.go 0.00% 23 Missing ⚠️
pkg/dataprotection/backup/request.go 4.54% 20 Missing and 1 partial ⚠️
pkg/dataprotection/restore/builder.go 0.00% 8 Missing and 2 partials ⚠️
pkg/dataprotection/backup/utils.go 84.61% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8693      +/-   ##
==========================================
+ Coverage   60.49%   61.39%   +0.90%     
==========================================
  Files         378      378              
  Lines       45623    47325    +1702     
==========================================
+ Hits        27598    29055    +1457     
- Misses      15441    15630     +189     
- Partials     2584     2640      +56     
Flag Coverage Δ
unittests 61.39% <36.49%> (+0.90%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@apecloud-bot apecloud-bot added the approved PR Approved Test label Dec 30, 2024
@apecloud-bot apecloud-bot removed the approved PR Approved Test label Dec 30, 2024
@shanshanying shanshanying added this to the Release 1.1 milestone Dec 30, 2024
@apecloud-bot apecloud-bot added the approved PR Approved Test label Dec 30, 2024
@apecloud-bot apecloud-bot removed the approved PR Approved Test label Jan 6, 2025
@gnolong gnolong requested a review from zjx20 January 6, 2025 09:30
@apecloud-bot apecloud-bot added the approved PR Approved Test label Jan 6, 2025
@gnolong gnolong merged commit 9bf690a into main Jan 6, 2025
37 checks passed
@gnolong gnolong deleted the support/incremental-backup branch January 6, 2025 11:47
@gnolong
Copy link
Contributor Author

gnolong commented Jan 7, 2025

/cherry-pick release-0.9

@apecloud-bot
Copy link
Collaborator

🤖 says: Error cherry-picking.

Auto-merging apis/dataprotection/v1alpha1/backup_types.go
Auto-merging apis/dataprotection/v1alpha1/backuppolicy_types.go
CONFLICT (modify/delete): apis/dataprotection/v1alpha1/backuppolicytemplate_types.go deleted in HEAD and modified in 9bf690a (feat: supports incremental backup (#8693)). Version 9bf690a (feat: supports incremental backup (#8693)) of apis/dataprotection/v1alpha1/backuppolicytemplate_types.go left in tree.
Auto-merging config/crd/bases/dataprotection.kubeblocks.io_backuppolicies.yaml
CONFLICT (modify/delete): config/crd/bases/dataprotection.kubeblocks.io_backuppolicytemplates.yaml deleted in HEAD and modified in 9bf690a (feat: supports incremental backup (#8693)). Version 9bf690a (feat: supports incremental backup (#8693)) of config/crd/bases/dataprotection.kubeblocks.io_backuppolicytemplates.yaml left in tree.
Auto-merging config/crd/bases/dataprotection.kubeblocks.io_backups.yaml
Auto-merging controllers/apps/transformer_cluster_backup_policy.go
CONFLICT (content): Merge conflict in controllers/apps/transformer_cluster_backup_policy.go
Auto-merging controllers/dataprotection/actionset_controller_test.go
CONFLICT (content): Merge conflict in controllers/dataprotection/actionset_controller_test.go
Auto-merging controllers/dataprotection/backup_controller.go
CONFLICT (content): Merge conflict in controllers/dataprotection/backup_controller.go
Auto-merging controllers/dataprotection/backup_controller_test.go
CONFLICT (content): Merge conflict in controllers/dataprotection/backup_controller_test.go
CONFLICT (modify/delete): controllers/dataprotection/backuppolicytemplate_controller_test.go deleted in HEAD and modified in 9bf690a (feat: supports incremental backup (#8693)). Version 9bf690a (feat: supports incremental backup (#8693)) of controllers/dataprotection/backuppolicytemplate_controller_test.go left in tree.
Auto-merging controllers/dataprotection/backupschedule_controller_test.go
CONFLICT (content): Merge conflict in controllers/dataprotection/backupschedule_controller_test.go
Auto-merging controllers/dataprotection/gc_controller_test.go
Auto-merging controllers/dataprotection/log_collection_controller_test.go
Auto-merging controllers/dataprotection/restore_controller_test.go
CONFLICT (content): Merge conflict in controllers/dataprotection/restore_controller_test.go
Auto-merging controllers/dataprotection/utils.go
Auto-merging deploy/helm/crds/dataprotection.kubeblocks.io_backuppolicies.yaml
CONFLICT (modify/delete): deploy/helm/crds/dataprotection.kubeblocks.io_backuppolicytemplates.yaml deleted in HEAD and modified in 9bf690a (feat: supports incremental backup (#8693)). Version 9bf690a (feat: supports incremental backup (#8693)) of deploy/helm/crds/dataprotection.kubeblocks.io_backuppolicytemplates.yaml left in tree.
Auto-merging deploy/helm/crds/dataprotection.kubeblocks.io_backups.yaml
Auto-merging docs/developer_docs/api-reference/backup.md
CONFLICT (content): Merge conflict in docs/developer_docs/api-reference/backup.md
Auto-merging pkg/dataprotection/backup/request.go
CONFLICT (content): Merge conflict in pkg/dataprotection/backup/request.go
Auto-merging pkg/dataprotection/backup/utils.go
Auto-merging pkg/dataprotection/restore/builder.go
Auto-merging pkg/dataprotection/restore/manager.go
CONFLICT (content): Merge conflict in pkg/dataprotection/restore/manager.go
Auto-merging pkg/dataprotection/restore/utils.go
Auto-merging pkg/dataprotection/utils/utils.go
CONFLICT (content): Merge conflict in pkg/dataprotection/utils/utils.go
Auto-merging pkg/testutil/dataprotection/backup_utils.go
Auto-merging pkg/testutil/dataprotection/constant.go
Auto-merging pkg/testutil/dataprotection/utils.go
CONFLICT (content): Merge conflict in pkg/testutil/dataprotection/utils.go
error: could not apply 9bf690a... feat: supports incremental backup (#8693)
hint: After resolving the conflicts, mark them with
hint: "git add/rm ", then run
hint: "git cherry-pick --continue".
hint: You can instead skip this commit with "git cherry-pick --skip".
hint: To abort and get back to the state before "git cherry-pick",
hint: run "git cherry-pick --abort".
hint: Disable this message with "git config advice.mergeConflict false"

@apecloud-bot
Copy link
Collaborator

🤖 says: ‼️ cherry pick action failed.
See: https://github.com/apecloud/kubeblocks/actions/runs/12644394874

@wangyelei
Copy link
Contributor

/cherry-pick release-1.0-beta

@apecloud-bot
Copy link
Collaborator

🤖 says: cherry pick action finished successfully 🎉!
See: https://github.com/apecloud/kubeblocks/actions/runs/12924008310

apecloud-bot pushed a commit that referenced this pull request Jan 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved PR Approved Test size/XXL Denotes a PR that changes 1000+ lines.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Features] supports incremental backup

7 participants