kranix-operator

Kubernetes operator — GitOps-native cluster reconciliation via custom resources.

kranix-operator runs inside your Kubernetes cluster as a controller watching Kranix's Custom Resource Definitions (CRDs). When you apply a KranixApp manifest to the cluster, the operator picks it up and drives the desired state through kranix-core. It is what makes Kranix GitOps-native: commit YAML, the operator reconciles. No manual kubectl apply chains, no out-of-band state.

---

What it does

• Installs and watches KranixApp, KranixNamespace, KranixPolicy, KranixPipeline, KranixSecret, and KranixDriftAlert CRDs
• Runs a Kubernetes controller loop using controller-runtime
• Translates CRD spec changes into kranix-core operations
• Reports reconciliation status back to the CRD's .status field
• Supports progressive delivery (canary and blue-green deployments)
• Supports multi-cluster federation for deploying across N clusters
• Supports multi-stage deployment pipelines via KranixPipeline CRD
• Supports automatic rollback on error rate or latency spikes
• Supports dependency ordering to ensure workloads start in the correct sequence
• Supports optional spec.cronSchedule on KranixApp for core-aligned periodic workloads
• Supports spec.scheduling (workload priority tiers, preemption / priorityClassName, spot / preemptible) and spec.crossNamespaceTraffic on KranixApp, forwarded toward core with the workload spec
• Provides admission webhooks for validation and mutation of resources before they are applied
• Syncs secrets from Vault, AWS Secrets Manager, and Azure Key Vault
• Detects and alerts on drift between Git state and cluster state
• Respects RBAC — operates only in namespaces it has been granted access to

---

Architecture position

Git repo  ──►  kubectl apply  ──►  Kubernetes API server
                                          │
                                    kranix-operator
                                          │
                                    kranix-core
                                          │
                                    kranix-runtime

The operator sits between the Kubernetes API server (where CRD state lives) and kranix-core (where orchestration logic lives). It is the GitOps bridge.

---

Custom resource definitions

KranixApp

The primary resource. Describes a workload you want Kranix to manage.

apiVersion: kranix.io/v1alpha1
kind: KranixApp
metadata:
  name: api-server
  namespace: production
spec:
  image: myorg/api-server:v1.4.2
  replicas: 3
  namespace: production
  env:
    DATABASE_URL: "postgres://..."
  resources:
    cpu: "500m"
    memory: "512Mi"
    gpu:
      vendor: "nvidia"           # nvidia | amd
      count: 2                   # Number of GPUs
      type: "A100"               # GPU type (optional)
      memory: "40Gi"             # GPU memory requirement (optional)
  ports:
    - containerPort: 8080
      protocol: TCP
  rollout:
    strategy: RollingUpdate
    maxUnavailable: 1
    healthCheckPath: /healthz
    healthCheckTimeout: 30s
  autoHeal: true
  # Optional — cron triggers in kranix-core; kranix-runtime uses a CronJob when backend is kubernetes.
  # cronSchedule:
  #   schedule: "0 * * * *"
  #   suspended: false
  #   timeZone: "UTC"
  #   concurrencyPolicy: Forbid     # Allow | Forbid | Replace
  metricsConfig:
    enabled: true
    collectionInterval: "30s"
    includeGPU: true
  progressiveDelivery:
    strategy: canary                # canary | blue-green | rolling
    canary:
      replicas: 1
      percentage: 10                # Percentage of traffic to canary (0-100)
      analysisDuration: "10m"
      successThreshold: 95
      metrics:
        - name: request_rate
          type: request_rate
          threshold: 1000
      autoPromote: true
      autoRollback: true
    blueGreen:
      previewReplicas: 2
      autoSwitch: true
      verificationDuration: "5m"
      verificationSteps:
        - name: http-check
          type: http
          config:
            url: http://app/healthz
          timeout: "30s"
          retryCount: 3
      rollbackOnFailure: true
  federation:
    enabled: true
    clusters:
      - name: cluster-east
        context: cluster-east
        weight: 50
        enabled: true
      - name: cluster-west
        context: cluster-west
        weight: 50
        enabled: true
    strategy: parallel              # parallel | sequential | rolling
    syncPolicy:
      syncInterval: "5m"
      autoSync: true
      conflictResolution: source-wins
    includeNetwork: true
    includeStorage: true

Example status (written by the operator controller):

status:
  phase: Running            # Pending | Deploying | Running | Degraded | Failed
  readyReplicas: 3
  lastReconciled: "2025-04-01T12:00:00Z"

The validating webhook rejects invalid cron expressions or time zones when cronSchedule is present and **suspended: false. The reconciler forwards cronSchedule toward kranix-core.

---

The KranixApp CRD supports progressive delivery strategies for safer deployments:

Canary Deployments

Gradually roll out new versions to a subset of traffic:

spec:
  progressiveDelivery:
    strategy: canary
    canary:
      replicas: 2
      percentage: 20              # 20% of traffic to canary
      analysisDuration: "15m"
      successThreshold: 95        # 95% success rate required
      metrics:
        - name: error_rate
          type: error_rate
          threshold: 0.01
        - name: latency_p95
          type: latency
          threshold: 500
      autoPromote: true
      autoRollback: true

Canary Features:

• Traffic splitting based on percentage
• Metric-based analysis during canary period
• Automatic promotion when thresholds are met
• Automatic rollback on failure
• Configurable analysis duration

Blue-Green Deployments

Deploy new version alongside current version with instant cutover:

spec:
  progressiveDelivery:
    strategy: blue-green
    blueGreen:
      previewReplicas: 3
      autoSwitch: true
      verificationDuration: "10m"
      verificationSteps:
        - name: health-check
          type: http
          config:
            url: http://app/healthz
            method: GET
          timeout: "30s"
          retryCount: 5
        - name: smoke-test
          type: script
          config:
            command: ./smoke-tests.sh
          timeout: "5m"
      rollbackOnFailure: true

Blue-Green Features:

• Zero-d deployments with instant traffic switch
• Configurable verification steps (HTTP, TCP, script, metric)
• Manual or automatic traffic switching
• Automatic rollback on verification failure
• Preview environment for testing

---

Multi-Cluster Federation

Deploy workloads across multiple Kubernetes clusters from a single control plane:

spec:
  federation:
    enabled: true
    clusters:
      - name: cluster-us-east
        context: cluster-us-east
        weight: 40
        replicas: 4
        regions:
          - us-east-1
        enabled: true
      - name: cluster-us-west
        context: cluster-us-west
        weight: 60
        replicas: 6
        regions:
          - us-west-2
        enabled: true
    strategy: parallel           # parallel | sequential | rolling
    syncPolicy:
      syncInterval: "5m"
      autoSync: true
      conflictResolution: source-wins  # last-write-wins | source-wins | manual
      resourceSelector:
        labelSelector:
          app: my-app
        includeTypes:
          - Deployment
          - Service
        excludeTypes:
          - ConfigMap

Federation Features:

• Deploy to N clusters from a single KranixApp
• Weight-based traffic routing across clusters
• Geographic region-based routing
• Per-cluster replica overrides
• Automatic sync with configurable intervals
• Conflict resolution strategies
• Resource filtering for selective sync

---

Multi-Stage Pipelines

Use KranixPipeline CRD to define complex deployment workflows:

apiVersion: kranix.io/v1alpha1
kind: KranixPipeline
metadata:
  name: production-deploy
spec:
  stages:
    - name: build
      order: 1
      type: deploy
      targetApp: my-app-build
      targetNamespace: build
      timeout: "20m"
    - name: test
      order: 2
      type: test
      config:
        testSuite: integration
      conditions:
        - type: previous_stage_success
          stage: build
      timeout: "30m"
      onFailure: stop
    - name: staging
      order: 3
      type: deploy
      targetApp: my-app
      targetNamespace: staging
      manualApproval: true
      timeout: "10m"
    - name: production
      order: 4
      type: promote
      targetApp: my-app
      targetNamespace: production
      conditions:
        - type: previous_stage_success
          stage: staging
      timeout: "5m"
      onFailure: rollback
  triggers:
    - type: git
      config:
        repository: github.com/myorg/my-app
        branch: main
      enabled: true
    - type: schedule
      config:
        cron: "0 2 * * *"
      enabled: false
  notifications:
    enabled: true
    channels:
      - type: slack
        config:
          webhook: https://hooks.slack.com/services/...
      - type: email
        config:
          recipients: team@company.com
    events:
      - started
      - completed
      - failed
      - stage_success
      - stage_failed
  timeout: "2h"
  retryPolicy:
    maxRetries: 3
    backoffDuration: "2m"
    retryOn:
      - network_error
      - timeout
  parallel: false

Pipeline Features:

• Multi-stage workflows with dependencies
• Stage conditions (previous stage success, metric thresholds)
• Manual approval gates
• Automatic retry with backoff
• Multiple trigger types (git, image, schedule, manual, webhook)
• Notification integrations (Slack, email, webhook, PagerDuty)
• Stage-level failure handling (stop, continue, rollback)
• Parallel or sequential execution

---

KranixNamespace

Declares a namespace that Kranix should manage:

apiVersion: kranix.io/v1alpha1
kind: KranixNamespace
metadata:
  name: staging
spec:
  labels:
    env: staging
  resourceQuota:
    cpu: "4"
    memory: "8Gi"

KranixPolicy

Defines infra policies applied to workloads in a namespace:

apiVersion: kranix.io/v1alpha1
kind: KranixPolicy
metadata:
  name: staging-policy
  namespace: staging
spec:
  enforceResourceLimits: true
  defaultCpuLimit: "500m"
  defaultMemoryLimit: "512Mi"
  allowPrivileged: false
  networkPolicy:
    ingressFrom: ["production"]

KranixPipeline

Defines multi-stage deployment pipelines for complex workflows:

apiVersion: kranix.io/v1alpha1
kind: KranixPipeline
metadata:
  name: deploy-pipeline
spec:
  stages:
    - name: deploy-staging
      order: 1
      type: deploy
      targetApp: my-app
      targetNamespace: staging
      timeout: "10m"
      onFailure: stop
    - name: test-staging
      order: 2
      type: test
      config:
        testType: integration
      conditions:
        - type: previous_stage_success
          stage: deploy-staging
      timeout: "15m"
      onFailure: stop
    - name: promote-production
      order: 3
      type: promote
      targetApp: my-app
      targetNamespace: production
      manualApproval: true
      timeout: "5m"
      onFailure: rollback
  triggers:
    - type: git
      config:
        branch: main
      enabled: true
    - type: image
      config:
        repository: myorg/my-app
      enabled: true
  notifications:
    enabled: true
    channels:
      - type: slack
        config:
          webhook: https://hooks.slack.com/...
    events:
      - started
      - completed
      - failed
  timeout: "1h"
  retryPolicy:
    maxRetries: 3
    backoffDuration: "1m"
  parallel: false
status:
  phase: Running               # Pending | Running | Succeeded | Failed | Paused
  currentStage: test-staging
  executionCount: 5
  lastExecution: "2025-04-01T12:00:00Z"
  stagesStatus:
    - name: deploy-staging
      phase: Succeeded
      started: "2025-04-01T12:00:00Z"
      completed: "2025-04-01T12:05:00Z"
    - name: test-staging
      phase: Running
      started: "2025-04-01T12:05:00Z"

Secret Synchronization

Sync secrets from external secret management systems to Kubernetes:

Vault Integration

apiVersion: kranix.io/v1alpha1
kind: KranixSecret
metadata:
  name: vault-db-credentials
spec:
  type: vault
  vault:
    address: https://vault.company.com
    path: secret/data/database
    engine: kv-v2
    authMethod: kubernetes
    authConfig:
      role: database-app
  destination:
    namespace: production
    name: db-credentials
  autoSync: true
  syncInterval: "5m"

AWS Secrets Manager

apiVersion: kranix.io/v1alpha1
kind: KranixSecret
metadata:
  name: aws-api-keys
spec:
  type: aws-secrets-manager
  awsSecretsManager:
    secretName: prod/api-keys
    region: us-east-1
  destination:
    namespace: production
    name: api-keys
  autoSync: true
  syncInterval: "10m"

Secret Sync Features:

• Support for HashiCorp Vault (KV v1/v2, database, etc.)
• Support for AWS Secrets Manager
• Support for Azure Key Vault
• Automatic sync with configurable intervals
• Data mapping for key transformation
• Multiple auth methods (token, Kubernetes, AWS IAM, etc.)

---

Drift Detection

Detect when cluster state diverges from Git state:

apiVersion: kranix.io/v1alpha1
kind: KranixDriftAlert
metadata:
  name: production-drift
spec:
  enabled: true
  checkInterval: "5m"
  alertOnDrift: true
  autoReconcile: true
  monitoredFields:
    - replicas
    - image
    - resources
  tolerance:
    replicaVariance: 0
    resourceVariancePct: 10.0
    envVarDriftAllowed: false
  gitConfig:
    repository: https://github.com/myorg/infrastructure
    branch: main
    path: manifests/production
  notificationHooks:
    - type: slack
      url: https://hooks.slack.com/services/...
    - type: pagerduty
      url: https://events.pagerduty.com/...

Drift Detection Features:

• Periodic comparison of cluster state vs Git state
• Configurable field monitoring
• Tolerance settings for acceptable variance
• Automatic reconciliation on drift detection
• Notification integrations (Slack, PagerDuty, webhooks)
• Support for multiple Git auth methods

---

KranixSecret

Syncs secrets from external secret management systems to Kubernetes:

apiVersion: kranix.io/v1alpha1
kind: KranixSecret
metadata:
  name: vault-secret
spec:
  type: vault                # vault | aws-secrets-manager | azure-key-vault
  vault:
    address: https://vault.example.com
    path: secret/data/my-app
    engine: kv-v2
    authMethod: kubernetes
    authConfig:
      role: my-app
      mount: /auth/kubernetes
    version: 2
  destination:
    namespace: production
    name: app-secrets
    type: Opaque
    dataMapping:
      password: DB_PASSWORD
      api-key: API_KEY
  syncInterval: "5m"
  autoSync: true
status:
  phase: Synced              # Pending | Syncing | Synced | Failed | Error
  lastSync: "2025-04-01T12:00:00Z"
  nextSync: "2025-04-01T12:05:00Z"
  syncCount: 42

KranixDriftAlert

Detects and alerts when cluster state diverges from Git state:

apiVersion: kranix.io/v1alpha1
kind: KranixDriftAlert
metadata:
  name: drift-monitor
spec:
  enabled: true
  checkInterval: "5m"
  alertOnDrift: true
  autoReconcile: false
  monitoredFields:
    - replicas
    - image
    - env
    - resources
  tolerance:
    replicaVariance: 0
    resourceVariancePct: 5.0
    envVarDriftAllowed: false
    labelDriftAllowed: true
    annotationDriftAllowed: true
  gitConfig:
    repository: https://github.com/myorg/infrastructure
    branch: main
    path: manifests/apps
    authMethod: token
    authConfig:
      token: git-token-secret
  notificationHooks:
    - type: slack
      url: https://hooks.slack.com/services/...
    - type: webhook
      url: https://webhook.example.com/drift
      headers:
        Authorization: Bearer token
status:
  phase: NoDrift             # Pending | Checking | NoDrift | Drifted | Reconciled
  lastCheck: "2025-04-01T12:00:00Z"
  nextCheck: "2025-04-01T12:05:00Z"
  severity: low
  message: No drift detected

---

Metrics Configuration

The KranixApp CRD supports resource usage metrics collection. Metrics can be configured in the metricsConfig field:

spec:
  metricsConfig:
    enabled: true
    collectionInterval: "30s"
    includeGPU: true
    includeNetwork: true
    includeStorage: true

Metrics Collected:

• CPU usage (cores and percentage of limit)
• Memory usage (bytes and percentage of limit)
• GPU metrics (utilization, memory, temperature, power) when includeGPU: true
• Network metrics (throughput, packets, errors) when includeNetwork: true
• Storage metrics (I/O, disk usage) when includeStorage: true

The operator configures kranix-runtime to collect these metrics and expose them to kranix-core for monitoring and auto-scaling decisions.

---

Dependency Ordering

The KranixApp CRD supports workload dependency ordering to ensure workloads start in the correct sequence. For example, you can declare that a database must be ready before the API server starts.

apiVersion: kranix.io/v1alpha1
kind: KranixApp
metadata:
  name: api-server
spec:
  image: myorg/api-server:v1.0.0
  replicas: 3
  namespace: production
  dependencies:
    - workloadId: database
      type: depends_on
      condition: ready
      timeout: "5m"
    - workloadId: cache
      type: waits_for
      condition: healthy
      timeout: "2m"

Dependency Types:

• depends_on - The workload depends on another workload to be in the specified condition
• waits_for - The workload waits for another workload to reach the specified condition
• requires - The workload requires another workload as a hard dependency

Dependency Conditions:

• running - The dependency workload is running
• healthy - The dependency workload is healthy (passing health checks)
• ready - The dependency workload is ready to serve traffic

Dependency Features:

• Automatic ordering of workload deployment based on dependencies
• Configurable timeout for each dependency
• Dependency tracking via labels (e.g., app.kranix.io/depends-on-0)
• Aligned with kranix-core's dependency resolution logic
• Supports circular dependency detection (in kranix-core)

---

Admission Webhooks

The operator provides admission webhooks for validating and mutating KranixApp resources before they are persisted in the cluster.

Validation Webhook:

• Validates required fields (image, namespace, replicas)
• Validates GPU configuration (vendor, count, type)
• Validates progressive delivery configuration
• Validates federation configuration
• Validates dependency configuration
• Validates auto-rollback configuration
• Ensures namespace names are valid
• Validates spec.cronSchedule when cron is not suspended (cron expression, timeZone, concurrencyPolicy)

Mutation Webhook:

• Sets default replicas to 1 if not specified
• Sets default rollout strategy to RollingUpdate if not specified
• Sets default progressive delivery strategy to rolling if not specified
• Adds app.kranix.io/managed=true label
• Adds dependency tracking labels (e.g., app.kranix.io/depends-on-0=database)

Webhook Configuration: Webhook manifests are located in config/webhook/ and include:

• kustomization.yaml - Kustomize configuration
• manifests.yaml - ValidatingWebhookConfiguration and MutatingWebhookConfiguration
• service.yaml - Webhook service definition
• patch_webhook.yaml - Namespace selector patches

Note: Admission webhooks are currently disabled due to controller-runtime API compatibility. The webhook logic is implemented and can be enabled after resolving the API version compatibility.

---

GPU Support

The KranixApp CRD supports GPU workload scheduling for both NVIDIA and AMD devices. GPU resources are specified in the resources.gpu field:

spec:
  resources:
    gpu:
      vendor: "nvidia"           # Required: "nvidia" or "amd"
      count: 2                   # Required: Number of GPUs to request
      type: "A100"               # Optional: GPU type/model (e.g., "A100", "V100", "MI250")
      memory: "40Gi"             # Optional: GPU memory requirement

GPU Resource Mapping:

• NVIDIA GPUs: Translated to nvidia.com/gpu resource in Kubernetes
• AMD GPUs: Translated to amd.com/gpu resource in Kubernetes
• The operator validates GPU vendor and ensures proper resource scheduling

Example GPU Workload:

apiVersion: kranix.io/v1alpha1
kind: KranixApp
metadata:
  name: ml-training
spec:
  image: myorg/ml-training:latest
  replicas: 1
  resources:
    cpu: "4"
    memory: "16Gi"
    gpu:
      vendor: "nvidia"
      count: 4
      type: "A100"
      memory: "80Gi"

---

Reconciliation loop

For each KranixApp the operator:

1. Reads the .spec from the Kubernetes API
2. Calls kranix-core with the desired workload spec
3. Core computes the diff and drives kranix-runtime
4. Operator watches for status events from core
5. Writes observed state back to .status on the CRD
6. Re-queues after the configured resync period

If a workload enters Degraded or Failed state and autoHeal: true is set, the operator triggers automatic remediation via core.

---

Project structure

kranix-operator/
├── cmd/
│   └── operator/                 # Entry point (controller-runtime manager)
├── internal/
│   ├── controllers/
│   │   ├── kranixapp_controller.go
│   │   ├── kranixnamespace_controller.go
│   │   └── kranixpolicy_controller.go
│   ├── reconciler/               # Reconciler logic (calls kranix-core)
│   ├── predicates/               # Event filter predicates
│   └── webhooks/                 # Admission webhooks (validation + mutation)
├── api/
│   └── v1alpha1/                 # CRD Go types + generated deepcopy
├── config/
│   ├── crd/                      # CRD YAML manifests (generated by controller-gen)
│   ├── rbac/                     # ClusterRole, ClusterRoleBinding
│   └── manager/                  # Deployment, ServiceAccount manifests
└── tests/
    ├── unit/
    └── e2e/                      # Uses envtest or a real cluster

---

Getting started

Prerequisites

• Go 1.22+
• controller-gen (go install sigs.k8s.io/controller-tools/cmd/controller-gen@latest)
• A Kubernetes cluster (kind, minikube, or real)
• kranix-core reachable from inside the cluster

Generate CRD manifests

controller-gen crd:trivialVersions=true rbac:roleName=kranix-operator-role \
  paths="./..." output:crd:artifacts:config=config/crd/bases

Install CRDs on your cluster

kubectl apply -f config/crd/bases/

Run locally (out-of-cluster)

git clone https://github.com/kranix-io/kranix-operator
cd kranix-operator
go mod download

KRANIX_CORE_ADDRESS=localhost:50051 \
go run ./cmd/operator --kubeconfig ~/.kube/config

Run tests

# Unit tests
go test ./internal/...

# E2E with envtest
go test ./tests/e2e/... -tags e2e

---

Deployment

The operator is deployed to the cluster via kranix-charts. Manual install:

kubectl apply -f config/rbac/
kubectl apply -f config/manager/

RBAC requirements

The operator's ServiceAccount needs the following cluster permissions:

rules:
  - apiGroups: ["kranix.io"]
    resources: ["kranixapps", "kranixnamespaces", "kranixpolicies"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
  - apiGroups: ["kranix.io"]
    resources: ["kranixapps/status"]
    verbs: ["get", "update", "patch"]
  - apiGroups: [""]
    resources: ["namespaces", "events"]
    verbs: ["get", "list", "watch", "create", "patch"]

---

Configuration

operator:
  resync_period: 30s
  max_concurrent_reconciles: 5
  leader_election: true
  leader_election_namespace: kranix-system

core:
  address: "kranix-core.kranix-system.svc.cluster.local:50051"

metrics:
  port: 8383

health:
  port: 8081

---

Connectivity

Repo	Relationship
kranix-core	Operator calls core for all reconciliation logic
kranix-charts	Operator is packaged and deployed via Helm charts
kranix-packages	Imports CRD types and shared utilities
Kubernetes API	Operator watches and updates CRDs via controller-runtime

---

Contributing

See CONTRIBUTING.md. CRD type changes require regenerating deepcopy and CRD YAML via controller-gen. All controllers must have E2E tests using envtest.

License

Apache 2.0 — see LICENSE.