Strimzi KafkaConnect arch migration

[jorgebo10]

Hi,
I'm migrating a Strimzi KafkaConnect cluster from an AMD64 nodegroup to an ARM64 (Graviton) nodegroup, and I'm running into a chicken-and-egg problem I'd like to understand better.

When I update the KafkaConnect resource with the new affinity/tolerations pointing to the ARM64 nodegroup, the following happens:

1. The existing AMD64 pod gets rescheduled onto the ARM64 node before a new image is built → exec format error → CrashLoopBackOff.
2. Once the pod is in CrashLoopBackOff, Strimzi marks the connector as NotReady and stops reconciling — so the build pod never gets triggered.
3. Even if I break out of that deadlock manually, the build pod doesn't trigger automatically just from the affinity change — I need to force a rebuild by annotating the StrimziPodSet.

My current workaround is: scale to 0 → patch affinity → force-rebuild annotation on StrimziPodSet → scale back to 1.

Is there a cleaner or officially supported way to handle this kind of arch migration with a single resource update, without hitting the CrashLoop/reconciliation deadlock in between?

Thanks,
Jorge

[scholzj]

I do not think there is any easy way how to achieve this to be honest. Sorry. I think the best way would be to use Image Volumes for the plugins which are platform independent or build a multiplatform image in your own CI instead of using Kafka Connect Build.

[jorgebo10]

It seems that updating everything at once, including the image URL change, does the trick. I'll continue testing. Thanks for the response.

spec:
  build:
    output:
      image: example-connect-image:latest-arm64
      pushSecret: registry-push-secret
      type: docker

    plugins:
      - artifacts:
          # plugin artifacts here

  template:
    buildPod:                         # Kaniko build pod
      tolerations:
        - key: example.io/node-role
          operator: Equal
          value: arm64-connect-node
          effect: NoSchedule

      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringIgnoredExecution:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: example.io/node-role
                    operator: In
                    values:
                      - arm64-connect-node

    pod:                               # Connect worker pod
      tolerations:
        - key: example.io/node-role
          operator: Equal
          value: arm64-connect-node
          effect: NoSchedule

      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: example.io/node-role
                    operator: In
                    values:
                      - arm64-connect-node

[scholzj]

Actualy you are right, that is true ... if you change the image name instead of using the force-rebuild, it should work.