Skip to content

Commit 02be94f

Browse files
committed
feat: add RBAC permissions and fix pod reconciler for DaemonSet mount mode
This commit completes the DaemonSet mount mode implementation with several critical fixes: RBAC Updates: - Add DaemonSet permissions (get, list, watch, create, update, patch) to CSI node service account - Update both k8s.yaml and k8s_before_v1_18.yaml deployment manifests - Enable CSI node pods to manage DaemonSets for mount pod lifecycle Pod Reconciler Improvements: - Add logic to detect mount mode transitions (shared-pod to DaemonSet) - Prevent recreation loops when switching between mount modes - Skip pod recreation when DaemonSet mode is active for a StorageClass - Properly clean up finalizers to allow seamless mode transitions Documentation: - Clarify all three mount modes: pvc, shared-pod, and daemonset - Document RBAC requirements for DaemonSet operations - Specify that nodeAffinity is only required for daemonset mode - Add comprehensive setup instructions including controller env variable requirement - Include example ConfigMap with proper mode configuration ConfigMap Template: - Update example mount-config.yaml with correct nodeAffinity selector - Show proper mode configuration for both default and specific StorageClasses These changes enable smooth operation of DaemonSet mount mode and proper handling of transitions between different mount modes without manual intervention.
1 parent 1df1d64 commit 02be94f

File tree

5 files changed

+102
-23
lines changed

5 files changed

+102
-23
lines changed

deploy/k8s.yaml

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -256,6 +256,17 @@ rules:
256256
- nodes
257257
verbs:
258258
- list
259+
- apiGroups:
260+
- apps
261+
resources:
262+
- daemonsets
263+
verbs:
264+
- get
265+
- list
266+
- watch
267+
- create
268+
- update
269+
- patch
259270
---
260271
apiVersion: rbac.authorization.k8s.io/v1
261272
kind: ClusterRole

deploy/k8s_before_v1_18.yaml

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -256,6 +256,17 @@ rules:
256256
- nodes
257257
verbs:
258258
- list
259+
- apiGroups:
260+
- apps
261+
resources:
262+
- daemonsets
263+
verbs:
264+
- get
265+
- list
266+
- watch
267+
- create
268+
- update
269+
- patch
259270
---
260271
apiVersion: rbac.authorization.k8s.io/v1
261272
kind: ClusterRole

deploy/kubernetes/csi-daemonset-mount/mount-config.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ apiVersion: v1
22
kind: ConfigMap
33
metadata:
44
name: juicefs-mount-config
5-
namespace: kube-system
5+
namespace: juicefs
66
data:
77
# Default configuration for all StorageClasses
88
default: |

docs/en/guide/daemonset-mount.md

Lines changed: 60 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -4,35 +4,71 @@ This feature allows JuiceFS CSI Driver to deploy Mount Pods as DaemonSets instea
44

55
## Overview
66

7-
When `STORAGE_CLASS_SHARE_MOUNT` is enabled, JuiceFS CSI Driver shares Mount Pods across multiple PVCs that use the same StorageClass. By default, these are created as individual Pods. With the DaemonSet option, Mount Pods are deployed as DaemonSets, providing:
7+
When `STORAGE_CLASS_SHARE_MOUNT` is enabled, JuiceFS CSI Driver shares Mount Pods across multiple PVCs that use the same StorageClass. By default, these are created as individual shared Pods. With the DaemonSet option, Mount Pods are deployed as DaemonSets, providing:
88

99
- **Better resource control**: DaemonSets ensure one Mount Pod per selected node
1010
- **Node affinity support**: Control which nodes run Mount Pods using nodeAffinity
1111
- **Automatic lifecycle management**: DaemonSets handle Pod creation/deletion automatically
1212
- **Simplified operations**: Easier to manage and monitor Mount Pods
1313
- **Works with existing StorageClasses**: No need to modify or recreate StorageClasses
14+
- **Automatic mode transition**: Seamlessly switches from shared-pod to DaemonSet mode
1415

15-
## Configuration
16+
## Prerequisites
17+
18+
### 1. Enable Mount Sharing
19+
20+
Set the `STORAGE_CLASS_SHARE_MOUNT` environment variable in **BOTH** the CSI Controller and CSI Node components:
21+
22+
```yaml
23+
# For StatefulSet (Controller)
24+
kubectl set env statefulset/juicefs-csi-controller -n kube-system STORAGE_CLASS_SHARE_MOUNT=true
25+
26+
# For DaemonSet (Node)
27+
kubectl set env daemonset/juicefs-csi-node -n kube-system STORAGE_CLASS_SHARE_MOUNT=true
28+
```
29+
30+
Or add to your Helm values:
31+
32+
```yaml
33+
node:
34+
storageClassShareMount: true
35+
controller:
36+
storageClassShareMount: true # Note: This may need to be added to the Helm chart
37+
```
1638
17-
### Enable DaemonSet Mount
39+
### 2. Grant RBAC Permissions
1840
19-
DaemonSet mount is automatically available when mount sharing is enabled. To enable mount sharing, set this environment variable in the CSI Driver deployment:
41+
The CSI Node service account needs permissions to manage DaemonSets. Add these permissions:
2042
2143
```yaml
22-
env:
23-
- name: STORAGE_CLASS_SHARE_MOUNT
24-
value: "true"
44+
- apiGroups:
45+
- apps
46+
resources:
47+
- daemonsets
48+
verbs:
49+
- get
50+
- list
51+
- watch
52+
- create
53+
- update
54+
- patch
2555
```
2656
27-
Once enabled, you can configure specific StorageClasses to use DaemonSet mode via the ConfigMap (see below).
57+
This is included in the latest deployment manifests (`deploy/k8s.yaml`).
58+
59+
## Configuration
60+
61+
### Mount Modes
2862

29-
### Configure Node Affinity
63+
JuiceFS CSI Driver supports three mount modes:
3064

31-
There are two ways to configure node affinity for DaemonSet Mount Pods:
65+
1. **`pvc`** (or `mountpod`): One mount pod per PVC - each PVC gets its own dedicated mount pod
66+
2. **`shared-pod`**: One shared mount pod per StorageClass per node - multiple PVCs share a mount pod
67+
3. **`daemonset`**: One DaemonSet for the entire StorageClass - ensures one mount pod per selected node
3268

33-
#### Method 1: ConfigMap (Recommended for existing StorageClasses)
69+
### Configure Mount Mode and Node Affinity
3470

35-
Create a ConfigMap to define node affinity for your StorageClasses without modifying them:
71+
Create a ConfigMap to configure mount mode and optionally node affinity:
3672

3773
```yaml
3874
apiVersion: v1
@@ -43,25 +79,27 @@ metadata:
4379
data:
4480
# Default configuration for all StorageClasses
4581
default: |
46-
nodeAffinity:
47-
requiredDuringSchedulingIgnoredDuringExecution:
48-
nodeSelectorTerms:
49-
- matchExpressions:
50-
- key: node-role.kubernetes.io/control-plane
51-
operator: DoesNotExist
82+
mode: shared-pod # Options: "pvc", "shared-pod", "daemonset"
83+
# nodeAffinity is NOT required for pvc or shared-pod modes
5284
5385
# Configuration for specific StorageClass by name
54-
my-existing-storageclass: |
55-
nodeAffinity:
86+
my-storageclass: |
87+
mode: daemonset
88+
nodeAffinity: # Required for daemonset mode to control which nodes run mount pods
5689
requiredDuringSchedulingIgnoredDuringExecution:
5790
nodeSelectorTerms:
5891
- matchExpressions:
59-
- key: juicefs/mount-node
92+
- key: node.kubernetes.io/workload
6093
operator: In
6194
values:
62-
- "true"
95+
- "compute"
6396
```
6497

98+
**Important notes:**
99+
- `mode`: Choose from `pvc`, `shared-pod`, or `daemonset`
100+
- `nodeAffinity`: Only required for `daemonset` mode to control which nodes the DaemonSet runs on
101+
- NOT required for `pvc` or `shared-pod` modes (pods are created on-demand where PVCs are used)
102+
65103
This method works with existing StorageClasses without any modifications.
66104

67105
#### Method 2: StorageClass Parameters (For new StorageClasses)

pkg/controller/pod_driver.go

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -462,6 +462,25 @@ func (p *PodDriver) podDeletedHandler(ctx context.Context, pod *corev1.Pod) (Res
462462

463463
// create
464464
if len(existTargets) != 0 && !hasAvailPod {
465+
// Check if we're using DaemonSet mode for this storage class
466+
// If so, don't recreate shared pods - let DaemonSet handle it
467+
if config.StorageClassShareMount && setting != nil {
468+
// Load mount configuration to check mode
469+
if err := config.LoadMountConfig(ctx, p.Client, setting); err != nil {
470+
log.Error(err, "Failed to load mount config, continuing with pod recreation")
471+
} else if config.ShouldUseDaemonSet(setting) {
472+
log.Info("Mount mode is DaemonSet, skipping pod recreation",
473+
"storageClass", setting.StorageClass,
474+
"uniqueId", setting.UniqueId)
475+
// Just remove finalizer and let DaemonSet handle the mount
476+
if err := resource.RemoveFinalizer(ctx, p.Client, pod, common.Finalizer); err != nil {
477+
log.Error(err, "remove pod finalizer error")
478+
return Result{}, err
479+
}
480+
return Result{}, nil
481+
}
482+
}
483+
465484
// create pod
466485
newPodName := podmount.GenPodNameByUniqueId(resource.GetUniqueId(*pod), true)
467486
log.Info("pod targetPath not empty, need to create a new one", "newPodName", newPodName)

0 commit comments

Comments
 (0)