gateway.networking.k8s.io_httproutes fail on GKE

[raphaelluchini]

What is the issue?

While creating a new cluster on GKE, gateway.networking.k8s.io HTTPRoute resource is already created. That conflicts with the linkerd-crds installed via helm. I am forced to disable using the enableHttpRoutes:false flag.

How can it be reproduced?

Bootstrap a new GKE cluster 1.27.3-gke.100
Install linkerd-crds 1.8.0 using helm. (In my case I used terraform)

Logs, error output, etc

helm_release.linkerd_crds: Creating...
╷
│ Error: rendered manifests contain a resource that already exists. Unable to continue with install: CustomResourceDefinition "httproutes.gateway.networking.k8s.io" in namespace "" exists and cannot be imported into the current release: invalid ownership metadata; label validation error: missing key "app.kubernetes.io/managed-by": must be set to "Helm"; annotation validation error: missing key "meta.helm.sh/release-name": must be set to "linkerd-crds"; annotation validation error: missing key "meta.helm.sh/release-namespace": must be set to "linkerd"

output of linkerd check -o short

Environment

GKE

Possible solution

I am not sure the impact on Linkerd operation. If has impact, Linkerd should create this CDR in a namespace. If has no impact, then just update the docs.

Additional context

No response

Would you like to work on fixing this bug?

no

[alpeb]

I created a standard cluster in GKE using the same version you posted, and I don't see the HTTPRoutes CRD in there. And to the best of my knowledge, that CRD isn't yet included by default in Kubernetes. Perhaps there's something else in your pipeline that is creating it? In that case, it makes sense to set enableHttpRoutes:false. Also note CRDs are cluster-level resources, so we can't create them under a namespace.

[dwilliams782]

Hi @alpeb - we are now facing this conflict as well. This CRD comes from having the Gateway API extensions installed in GKE: https://cloud.google.com/kubernetes-engine/docs/concepts/gateway-api

I'm quite concerned about this. We use Linkerd primarily for unified metrics, and all the other features are just a bonus. With HTTPRoutes likely deprecating ServiceProfiles in the future, I'm concerned that we will be faced with a decision between GKE's Gateway API and Linkerd.

[raphaelluchini]

@alpeb Thanks for clarifying. In fact @dwilliams782 is correct. I forgot to mention the Gateway API was enabled.
Linkerd HTTPRoutes has different version from Google's as well. As we use Google's Gateway API, for now we choose Google's HTTPRoutes.

[kflynn]

To back up – the issue you're seeing is that you had GKE install Gateway API, and then the linkerd-crds chart failed, correct?

[kflynn]

Doublechecking with GKE engineering, telling the cluster you want Gateway API is an opt-in thing. @alpeb, we should talk about that a bit – I do expect the linkerd-crds chart to fail on such a cluster. 🙁

[alpeb]

Thank you all for the clarifications. Note however that we do have the enableHttpRoutes option in the linkerd-crds chart, which should be set to false in this case. This is about the the same CRD so it shouldn't be a problem to use GKE's, but folks should be aware they might get less features support depending on the version GKE is giving them. We probably need to document those versions in our reference documentation.

[kflynn]

Thanks @alpeb! I'll take the docs item (linkerd/website#1711), and will close this issues since --set enableHttpRoutes=false on the Helm install should take care of the issue. 🙂