vsphere-csi-driver:2.7.0 is failing on ROKS 4.13 cluster

[gauravAggrahari]

Currently vsphere-csi-driver:2.70 is not compatible with ROKS 4.13.
ROKS 4.13 support K8S 1.26 where as the max version supported by current compatibility matrix is 1.25

Compatibility Matrix:

"CSI" : {
    "2.7.0" : {
      "vSphere": { "min": "6.7.1", "max": "8.0.1" },
      "k8s": { "min": "1.23", "max": "1.25" },
      "isCPIRequired": false,
      "deploymentPath": [
          "https://raw.githubusercontent.com/vmware-tanzu/vsphere-kubernetes-drivers-operator/release/artifacts/csi/namespace.yaml",
          "https://raw.githubusercontent.com/vmware-tanzu/vsphere-kubernetes-drivers-operator/release/artifacts/csi/webhook.yaml",
          "https://raw.githubusercontent.com/vmware-tanzu/vsphere-kubernetes-drivers-operator/release/artifacts/csi/vsphere-csi-driver.yaml" ]
    }

Also there is a need to refresh the template as it was failing with the below errors.

Replicaset Events

Events:
  Type     Reason        Age                  From                   Message
  ----     ------        ----                 ----                   -------
  Warning  FailedCreate  2m30s                replicaset-controller  Error creating: pods "vdo-controller-manager-588498fc6c-kkkl7" is forbidden: violates PodSecurity "baseline:latest": host namespaces (hostNetwork=true), hostPath volumes (volume "vsphere-config-volume"), hostPort (container "kube-rbac-proxy" uses hostPort 8443)
  Warning  FailedCreate  2m30s                replicaset-controller  Error creating: pods "vdo-controller-manager-588498fc6c-cs26w" is forbidden: violates PodSecurity "baseline:latest": host namespaces (hostNetwork=true), hostPath volumes (volume "vsphere-config-volume"), hostPort (container "kube-rbac-proxy" uses hostPort 8443)
  Warning  FailedCreate  2m30s                replicaset-controller  Error creating: pods "vdo-controller-manager-588498fc6c-62hc9" is forbidden: violates PodSecurity "baseline:latest": host namespaces (hostNetwork=true), hostPath volumes (volume "vsphere-config-volume"), hostPort (container "kube-rbac-proxy" uses hostPort 8443)
  Warning  FailedCreate  2m30s                replicaset-controller  Error creating: pods "vdo-controller-manager-588498fc6c-xlw5p" is forbidden: violates PodSecurity "baseline:latest": host namespaces (hostNetwork=true), hostPath volumes (volume "vsphere-config-volume"), hostPort (container "kube-rbac-proxy" uses hostPort 8443)
  Warning  FailedCreate  2m30s                replicaset-controller  Error creating: pods "vdo-controller-manager-588498fc6c-bj946" is forbidden: violates PodSecurity "baseline:latest": host namespaces (hostNetwork=true), hostPath volumes (volume "vsphere-config-volume"), hostPort (container "kube-rbac-proxy" uses hostPort 8443)
  Warning  FailedCreate  2m30s                replicaset-controller  Error creating: pods "vdo-controller-manager-588498fc6c-wlpz5" is forbidden: violates PodSecurity "baseline:latest": host namespaces (hostNetwork=true), hostPath volumes (volume "vsphere-config-volume"), hostPort (container "kube-rbac-proxy" uses hostPort 8443)
  Warning  FailedCreate  2m30s                replicaset-controller  Error creating: pods "vdo-controller-manager-588498fc6c-fjc6r" is forbidden: violates PodSecurity "baseline:latest": host namespaces (hostNetwork=true), hostPath volumes (volume "vsphere-config-volume"), hostPort (container "kube-rbac-proxy" uses hostPort 8443)
  Warning  FailedCreate  2m29s                replicaset-controller  Error creating: pods "vdo-controller-manager-588498fc6c-zrvjw" is forbidden: violates PodSecurity "baseline:latest": host namespaces (hostNetwork=true), hostPath volumes (volume "vsphere-config-volume"), hostPort (container "kube-rbac-proxy" uses hostPort 8443)
  Warning  FailedCreate  2m29s                replicaset-controller  Error creating: pods "vdo-controller-manager-588498fc6c-qgh7s" is forbidden: violates PodSecurity "baseline:latest": host namespaces (hostNetwork=true), hostPath volumes (volume "vsphere-config-volume"), hostPort (container "kube-rbac-proxy" uses hostPort 8443)
  Warning  FailedCreate  68s (x6 over 2m27s)  replicaset-controller  (combined from similar events): Error creating: pods "vdo-controller-manager-588498fc6c-2db7z" is forbidden: violates PodSecurity "baseline:latest": host namespaces (hostNetwork=true), hostPath volumes (volume "vsphere-config-volume"), hostPort (container "kube-rbac-proxy" uses hostPort 8443)

After running the below commands the errors disappeared

kubectl label --overwrite ns vmware-system-vdo pod-security.kubernetes.io/enforce=privileged
kubectl label --overwrite ns vmware-system-csi pod-security.kubernetes.io/enforce=privileged

But the VDO logs had these errors
vdo pod logs

I0326 08:55:57.485207       1 vdoconfig_controller.go:1209] controllers/VDOConfig "msg"="vSphere Versions "  "version"=["7.0.3"]
I0326 08:55:57.485224       1 vdoconfig_controller.go:1210] controllers/VDOConfig "msg"="k8s Versions "  "version"="1.26"
E0326 08:55:57.485286       1 vdoconfig_controller.go:1355] controllers/VDOConfig "msg"="Error occurred when fetching the CSI deployment yamls" "error"="could not fetch compatible CSI version for vSphere version and k8s version "
E0326 08:55:57.485321       1 controller.go:302] controller-runtime/manager/controller/vdoconfig "msg"="Reconciler error" "error"="could not fetch compatible CSI version for vSphere version and k8s version " "name"="vdo-configbfd24d6a" "namespace"="vmware-system-vdo" "reconciler group"="vdo.vmware.com" "reconciler kind"="VDOConfig"