Able to migrate volumes to another storage class and safer mechanism to prevent paths to be discovered again

[cofyc]

Motivation

Able to migrate PVs (actually underlying volumes) to another storage class and do not be afraid volumes can possibly be discovered more than once.

Problem

Current PV name (generated from path + class + node) is used as a unique identifier for the path on the node. It has some drawbacks, e.g. cannot change PV naming convention anymore, and users cannot change the storage class name, otherwise, volumes will be discovered again.

For example, at first, a user creates a storage class foo to discovery volumes under /mnt/disks, but find foo name is not a good name and want to change it or simply changed the configuration by accident, volumes under /mnt/disks will be discovered again and new PVs with same paths will be created. Pods which expect to use different PVs under different storage class may mount the same volume.

Solution

• Check PV.spec.path and PV.nodeAffinity, if there is a matching PV in PV cache, skip discovering it
• A discovery path can be configured under more than one storage class, but only one can be enabled for discovering
  • Note that all storage classes are able to be enabled for deleting

Benefits:

• Do not depend on PV name, we can use any naming mechanism in future
• It helps the user to migrate volumes to new storage class

Use cases

Name of storage class is changed by accident

In the beginning, alice as system admin uses the following configuration for provisioner:

    foo:
      hostDir: /mnt/disks
      mountDir: /mnt/disks

Some volumes are discovered for storage class foo. Alice changes the storage class name to bar in provisioner configuration:

    bar:
      hostDir: /mnt/disks
      mountDir: /mnt/disks

Volumes under storage class foo will not be created under storage class bar. Only new volumes added in /mnt/disks will be created.

This is safer than the current behavior.

Note that, old volumes under storage class foo cannot be deleted by provisioner in this scenario because current provisioner will skip if storage class is not found in its configuration. (For in-process deleting, provisioner has no way to know mount path in provisioner container without storage class configuration) However, users can add old configuration back to recover deletion.

Migrate volumes to new storage class

In the beginning, alice as system admin uses the following configuration for provisioner:

    foo:
      hostDir: /mnt/disks
      mountDir: /mnt/disks

Alice wants to migrate volumes under /mnt/disks to another storage class bar, she can add a new storage class:

    foo:
      hostDir: /mnt/disks
      mountDir: /mnt/disks
      disabledForDiscovery: true # only one storage class for same `hostDir` can be active for discovery (default)
    bar:
      hostDir: /mnt/disks
      mountDir: /mnt/disks

Provisioner now can discover new volumes from /mnt/disks under storage class bar.

When volumes under foo are released and marked persistentVolumeReclaimPolicy=Delete, they will be deleted by provisioner.

When all volumes under foo are deleted, foo storage class can be removed from provisioner configuration. Of course, this is optional.

    bar:
      hostDir: /mnt/disks
      mountDir: /mnt/disks

[cofyc]

cc @msau42 What do you think?

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[msau42]

/remove-lifecycle stale

We might need something more stable than device name + node name. Maybe serial number? There is an ask to better support migration of local volumes to new nodes. #127

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[msau42]

/lifecycle frozen

[jinjianming]

Any progress made