NetApp Trident CSI Integration for Enterprise RAG

This document describes the NetApp Trident CSI driver integration implemented for the Intel Enterprise RAG project to support AIPod Mini deployments with NetApp ONTAP storage.

Overview

NetApp Trident is a dynamic storage orchestrator for Kubernetes that enables persistent storage for containerized applications using NetApp storage systems. This integration adds support for NetApp ONTAP storage as a CSI driver option in the Enterprise RAG deployment.

Integration Summary

Purpose

Integrate NetApp Trident CSI deployment automation into the Intel Enterprise RAG project to provide enterprise-grade persistent storage capabilities using NetApp ONTAP systems.

Key Features

Automated Trident operator installation using Helm
ONTAP NAS backend configuration
Dynamic StorageClass creation with ReadWriteMany support
Seamless integration with existing Enterprise RAG deployment workflow
Ubuntu-specific NFS utilities installation for Trident prerequisites

Current Project Structure

Enterprise RAG Deployment Architecture

The Enterprise RAG project follows a structured Ansible-based deployment approach with the following key components:

Enterprise-RAG/
├── deployment/
│   ├── playbooks/
│   │   ├── infrastructure.yaml           # Infrastructure setup playbook
│   │   ├── application.yaml              # Application deployment playbook
│   │   └── validate.yaml                 # Validation playbook
│   ├── roles/
│   │   ├── application/                  # Application-specific roles
│   │   ├── common/                       # Shared utilities and validation
│   │   │   └── validate_config/          # Configuration validation
│   │   └── infrastructure/               # Infrastructure roles
│   │       ├── cluster/                  # Kubernetes cluster management
│   │       │   └── tasks/
│   │       │       └── post_installation.yaml  # Post-install tasks
│   │       ├── gaudi_operator/           # Intel Gaudi support
│   │       ├── k8s_local_registry/       # Local container registry
│   │       ├── nfs_server_csi_setup/     # NFS CSI driver (existing)
│   │       └── velero/                   # Backup solution
│   └── inventory/
│       └── sample/
│           └── config.yaml               # Sample configuration file
├── docs/                                 # Project documentation
└── src/                                  # Source code components

Current CSI Driver Support

Before Trident integration, Enterprise RAG supported:

local-path-provisioner: For single-node deployments
- Location: Built into Kubespray
- Use case: Development and single-node setups
- Access modes: ReadWriteOnce only
nfs: For multi-node deployments
- Location: deployment/roles/infrastructure/nfs_server_csi_setup/
- Use case: Multi-node clusters requiring ReadWriteMany
- Implementation: In-cluster NFS server + CSI driver

Configuration Flow

User Configuration: Users modify inventory/sample/config.yaml
Validation: roles/common/validate_config/ validates settings
Infrastructure Setup: playbooks/infrastructure.yaml handles Kubernetes and CSI setup
Post-Installation: roles/infrastructure/cluster/tasks/post_installation.yaml manages CSI drivers
Application Deployment: playbooks/application.yaml deploys Enterprise RAG components

Files Modified/Created

1. Infrastructure Playbook

File: deployment/playbooks/infrastructure.yaml

Changes Made:

Added NFS utilities installation section for Ubuntu systems
Conditional installation when install_csi == "netapp-trident"
Installs nfs-common and nfs-kernel-server packages for NFS client utilities and NFS server utilities (required for Trident NFS backends) respectively.

- name: Install NFS utilities
  hosts: k8s_cluster
  become: true
  tags:
    - install
    - post-install
  tasks:
    - name: Install NFS utilities on Ubuntu
      ansible.builtin.package:
        name:
          - nfs-common
          - nfs-kernel-server
        state: present
        update_cache: true
      when: 
        - ansible_distribution == "Ubuntu"
        - install_csi == "netapp-trident"

2. Post-Installation Tasks

File: deployment/roles/infrastructure/cluster/tasks/post_installation.yaml

Changes Made:

Added NetApp Trident CSI role inclusion for installation
Added uninstall task for cleanup operations

- name: NetApp Trident CSI role
  ansible.builtin.include_role:
    name: netapp_trident_csi_setup
  when: install_csi == "netapp-trident"
  tags:
    - install
    - post-install

- name: Uninstall NetApp Trident
  ansible.builtin.include_role:
    name: netapp_trident_csi_setup
  when: install_csi == "netapp-trident"
  tags:
    - delete

3. Configuration Validation

File: deployment/roles/common/validate_config/tasks/main.yaml

Changes Made:

Updated CSI driver validation to include "netapp-trident" as valid option

# Before
install_csi not in ['local-path-provisioner', 'nfs']

# After  
install_csi not in ['local-path-provisioner', 'nfs', 'netapp-trident']

4. Sample Configuration

File: deployment/inventory/sample/config.yaml

Changes Made:

Added "netapp-trident" to available CSI options documentation
Added comprehensive Trident configuration section with ONTAP parameters

# Available options:
# - "netapp-trident": Use for NetApp ONTAP storage with Trident CSI driver

# Setup when install_csi is "netapp-trident"
trident_operator_version: "2510.0"
trident_namespace: "trident"
trident_storage_class: "netapp-trident"
trident_backend_name: "ontap-nas"
ontap_management_lif: ""
ontap_data_lif: ""
ontap_svm: ""
ontap_username: ""
ontap_password: ""

5. NetApp Trident CSI Role Tasks

File: deployment/roles/infrastructure/netapp_trident_csi_setup/tasks/main.yaml

Changes Made:

Added ONTAP connectivity validation for both management LIF and data LIF before attempting backend registration
Added verification that the TridentBackendConfig object reaches Bound phase after creation, failing fast with a clear error if the backend does not become bound within the timeout

- name: Verify ONTAP management LIF connectivity
  ansible.builtin.wait_for:
    host: "{{ ontap_management_lif }}"
    port: 443
    timeout: 10
  register: mgmt_lif_check
  failed_when: mgmt_lif_check.failed
  tags:
    - install
    - post-install

- name: Verify ONTAP data LIF connectivity
  ansible.builtin.wait_for:
    host: "{{ ontap_data_lif }}"
    port: 2049
    timeout: 10
  register: data_lif_check
  failed_when: data_lif_check.failed
  tags:
    - install
    - post-install

- name: Wait for TridentBackendConfig to reach Bound phase
  kubernetes.core.k8s_info:
    api_version: trident.netapp.io/v1
    kind: TridentBackendConfig
    name: "{{ trident_backend_name }}"
    namespace: "{{ trident_namespace }}"
  register: backend_status
  until: backend_status.resources[0].status.phase == "Bound"
  retries: 12
  delay: 10
  failed_when: backend_status.resources[0].status.phase != "Bound"
  tags:
    - install
    - post-install

New Ansible Role Created

Role Structure

deployment/roles/infrastructure/netapp_trident_csi_setup/
├── defaults/main.yaml
├── tasks/main.yaml
├── templates/
│   ├── trident-backend.yaml.j2
│   └── trident-storageclass.yaml.j2
└── README.md

Role Components

defaults/main.yaml

Default variables for Trident configuration including:

Trident operator version (25.10)
Namespace and StorageClass names
ONTAP backend connection parameters

tasks/main.yaml

Main Ansible tasks including:

Install Tasks:
- Create Trident namespace
- Add official NetApp Helm repository
- Install Trident operator via Helm
- Validate connectivity to ONTAP management LIF (port 443)
- Validate connectivity to ONTAP data LIF (port 2049 / NFS)
- Create ONTAP NAS backend configuration
- Verify TridentBackendConfig object reaches Bound phase
- Create StorageClass with ReadWriteMany support
- Manage default StorageClass annotations
Delete Tasks:
- Uninstall Trident operator
- Remove Trident namespace

Templates

trident-backend.yaml.j2: TridentBackendConfig CRD for ONTAP NAS

apiVersion: trident.netapp.io/v1
kind: TridentBackendConfig
metadata:
  name: {{ trident_backend_name }}
  namespace: {{ trident_namespace }}
spec:
  version: 1
  storageDriverName: ontap-nas
  managementLIF: {{ ontap_management_lif }}
  dataLIF: {{ ontap_data_lif }}
  svm: {{ ontap_svm }}
  username: {{ ontap_username }}
  password: {{ ontap_password }}
  useREST: true

trident-storageclass.yaml.j2: Kubernetes StorageClass

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: {{ trident_storage_class }}
  annotations:
    storageclass.kubernetes.io/is-default-class: "true"
provisioner: csi.trident.netapp.io
parameters:
  backendType: ontap-nas
  fsType: nfs
allowVolumeExpansion: true
volumeBindingMode: WaitForFirstConsumer
reclaimPolicy: Delete

Configuration Requirements

Prerequisites

NetApp ONTAP System: Configured ONTAP cluster with:
- ONTAP software version 9.16.1P4 or above
- Storage Virtual Machine (SVM) configured
- NFS protocol enabled
- Data and management LIFs configured
- Aggregate with available space
Kubernetes Cluster: Running Kubernetes cluster with:
- Helm 3.17 or higher installed
- Ubuntu nodes (for NFS utilities installation)
- Network connectivity to ONTAP system
Authentication: ONTAP credentials with administrative privileges

Required Configuration Parameters

Users must provide the following ONTAP-specific parameters in their config.yaml:

install_csi: "netapp-trident"
ontap_management_lif: "192.168.1.100"    # ONTAP management interface IP
ontap_data_lif: "192.168.1.101"          # ONTAP data interface IP
ontap_svm: "svm_ai"                       # Storage Virtual Machine name
ontap_username: "admin"                   # ONTAP admin username
ontap_password: "password123"             # ONTAP admin password

# Required: route storage traffic through the ingress reverse proxy so that
# web browsers (which cannot reach the ONTAP data_lif directly) can upload files.
reverse_proxy_storage: true

# Required EDP configuration when using ONTAP:
# - rbac must be disabled
# - storageType must be s3compatible
# - externalUrl must point to the ingress-exposed S3 hostname
edp:
  enabled: true
  rbac:
    enabled: false    # Must be disabled when using ONTAP
  storageType: s3compatible
  s3compatible:
    internalUrl: "https://192.168.1.101"   # ONTAP data_lif – used by in-cluster pods
    externalUrl: "https://s3.erag.com"     # Ingress hostname – used by web browsers

Note: externalUrl must match the hostname registered in your DNS and ingress configuration (e.g. s3.<your-fqdn>). This is the address browsers use when uploading files through the UI; the data_lif IP is only reachable inside the cluster.

Step 1: Configure Parameters

Copy sample configuration:

cp -r inventory/sample inventory/my-cluster

Edit inventory/my-cluster/config.yaml:
- Set install_csi: "netapp-trident"
- Fill in all ONTAP connection parameters
- Set reverse_proxy_storage: true to enable the ingress reverse proxy for storage traffic (required because web browsers cannot reach the ONTAP data_lif directly)
- Set edp.s3compatible.externalUrl to https://s3.<your-fqdn> (e.g. https://s3.erag.com) — this is the ingress-exposed S3 hostname used by the web UI; the data_lif IP is not accessible from outside the cluster
- Set edp.rbac.enabled: false — must be disabled when using ONTAP as the storage backend
- Configure other Enterprise RAG settings

Step 2: Deploy Infrastructure

ansible-playbook -K playbooks/infrastructure.yaml \
  --tags post-install \
  -i inventory/my-cluster/inventory.ini \
  -e @inventory/my-cluster/config.yaml

Step 3: Deploy Enterprise RAG Application

ansible-playbook playbooks/application.yaml \
  -i inventory/my-cluster/inventory.ini \
  -e @inventory/my-cluster/config.yaml

Storage Capabilities

ReadWriteMany Support

The Trident CSI driver with ONTAP NAS backend provides:

ReadWriteMany (RWX): Multiple pods can mount the same volume with read-write access
ReadWriteOnce (RWO): Single pod exclusive access
ReadOnlyMany (ROX): Multiple pods with read-only access

Dynamic Provisioning

Automatic volume creation on-demand
Volume expansion support
Snapshot capabilities (with VolumeSnapshotClass)
Backup integration with Velero

Integration Benefits

For Enterprise RAG

Multi-node Support: RWX capability enables pod scheduling across multiple nodes
Enterprise Storage: Production-grade NetApp ONTAP storage backend
Data Protection: Built-in ONTAP features (snapshots, replication, backup)
Performance: High-performance NFS storage for AI workloads
Scalability: Dynamic volume provisioning and expansion

For AIPod Mini

Simplified Deployment: Automated Trident installation and configuration
Consistent Storage: Standardized storage across AIPod deployments
Enterprise Integration: Seamless integration with existing NetApp infrastructure
Support Matrix: Supported configuration for enterprise deployments

Troubleshooting

Common Issues

NFS Utilities Installation Fails
- Ensure nodes are running Ubuntu
- Check network connectivity for package downloads
- Verify sudo/root privileges
Trident Installation Fails
- Verify Helm repository accessibility
- Check Kubernetes cluster connectivity
- Ensure sufficient cluster resources
Backend Configuration Fails
- Verify ONTAP connectivity (management and data LIFs)
- Check ONTAP credentials and permissions
- Ensure SVM and aggregate exist
- Verify NFS protocol is enabled on SVM
StorageClass Issues
- Check if Trident CSI driver pods are running
- Verify backend is registered with Trident
- Check for conflicting default StorageClasses

Verification Commands

# Check Trident installation
kubectl get pods -n trident

# Check StorageClass
kubectl get storageclass

# Check Trident backends
kubectl get tridentbackendconfig -n trident

# Check volume provisioning
kubectl get pvc --all-namespaces

Security Considerations

Credential Management: ONTAP credentials are stored in Kubernetes secrets
Network Security: Ensure secure communication between Kubernetes and ONTAP
RBAC: Implement proper Kubernetes RBAC for Trident resources
Storage Security: Configure ONTAP security features (encryption, access controls)

Version Compatibility

Trident Version: 25.10 (Helm chart version 100.2510.0)
Kubernetes: 1.31+ (as supported by Enterprise RAG)
ONTAP: 9.16.1P4+ (recommended for full feature support)
Ubuntu: 22.04/24.04 (for NFS utilities)

Future Enhancements

Potential areas for future development:

Additional Storage Drivers: Support for ONTAP SAN (iSCSI, FC)
Advanced Features: Integration with ONTAP snapshots and clones
Performance Tuning: Optimized configurations for AI workloads
Monitoring: Integration with NetApp monitoring tools
GenAI Examples: Sample applications demonstrating Trident storage usage

Support and Documentation

NetApp Trident Documentation: https://docs.netapp.com/us-en/trident/
Helm Installation Guide: https://docs.netapp.com/us-en/trident/trident-get-started/kubernetes-deploy-helm.html
Enterprise RAG Documentation: See main project README.md

This integration was developed as part of NetApp's contribution to the OPEA Enterprise RAG project in support of AIPod Mini deployments.

Uh oh!

FilesExpand file tree

netapp_trident_integration.md

Latest commit

History

netapp_trident_integration.md

File metadata and controls

NetApp Trident CSI Integration for Enterprise RAG

Overview

Integration Summary

Purpose

Key Features

Current Project Structure

Enterprise RAG Deployment Architecture

Current CSI Driver Support

Configuration Flow

Files Modified/Created

1. Infrastructure Playbook

2. Post-Installation Tasks

3. Configuration Validation

4. Sample Configuration

5. NetApp Trident CSI Role Tasks

New Ansible Role Created

Role Structure

Role Components

defaults/main.yaml

tasks/main.yaml

Templates

Configuration Requirements

Prerequisites

Required Configuration Parameters

Step 1: Configure Parameters

Step 2: Deploy Infrastructure

Step 3: Deploy Enterprise RAG Application

Storage Capabilities

ReadWriteMany Support

Dynamic Provisioning

Integration Benefits

For Enterprise RAG

For AIPod Mini

Troubleshooting

Common Issues

Verification Commands

Security Considerations

Version Compatibility

Future Enhancements

Support and Documentation