fix(kubeadm): preserve kubelet.conf server line indent via backrefs

[mehrdadbn9]

Description

The "Update server field in kubelet kubeconfig" tasks rewrote the server: line with a fixed indent. When the on-disk kubelet.conf uses a different indent than assumed, server: lands outside the cluster: mapping and kubelet fails after upgrade with:

invalid configuration: no server found for cluster "default-cluster"

and worker nodes crash-loop (NotReady, tainted unreachable).

This switches both variants (default and external-lb) to backrefs: true with regexp: '^(\s*)server: .*$' and line: '\1server: {{ kube_apiserver_endpoint }}', so the existing indent is captured and preserved while only the endpoint value is replaced.

Refs #13277

Verification

Reproduced with real ansible-playbook 2.16 lineinfile before pushing: without backrefs: true the literal text \1server: is written (invalid YAML, parse error); with backrefs, both 4-space and 8-space indents are preserved, the endpoint is substituted, and the CA line is untouched.

Type of change

• Bug fix (non-breaking change which fixes an issue)

Fix kubelet.conf `server:` handling to preserve the existing indentation when updating the endpoint, so worker nodes do not crash-loop after upgrade (kubernetes-sigs/kubespray#13277).

[kubernetes-prow]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: mehrdadbn9
Once this PR has been reviewed and has the lgtm label, please assign mzaian for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[kubernetes-prow]

Hi @mehrdadbn9. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[yankay]

/ok-to-test

[guoard]

I don't think kubeadm output actually changed. v1.35/v1.36 still write kubelet.conf via clientcmd.WriteToFile (sigs.k8s.io/yaml v1.6.0, goyaml.v2 encoder), so server: is still at 4 spaces. The samples in #13277 don't look like real kubeadm output either.

Against a real kubelet.conf this patch breaks things: the first task replaces the 4-space line with an 8-space one (YAML parse error next to certificate-authority-data:), and the new external lb regexp never matches, so lineinfile appends the line at EOF.

If we want to make this more robust, preserving the existing indent would be better than hardcoding it:

regexp: '^(\s*)server: .*$'
line: '\1server: {{ kube_apiserver_endpoint }}'
backrefs: true

Would be good to get an unedited kubelet.conf from the reporter of #13277 first.

[mehrdadbn9]

You were right about the previous attempt. Two real bugs, both reproduced locally with ansible-playbook 2.16 lineinfile before this push:

1. The rewrite on 34d3fe5 kept '\1' in line but missed backrefs: true on both tasks. lineinfile then writes the literal string '\1server: ...' and kubelet.conf becomes invalid YAML (same failure mode you described).

Repro:

• without backrefs: literal '\1server: https://LB-VIP:6443' written, yaml parse error line 6
• with backrefs: indent preserved (4-space and 8-space tested), certificate-authority-data untouched

2. New head ed7bd07 adds backrefs: true to both tasks and tightens the regexp to '^(\s*)server: .*$' so it can only match a server: key with a value, not a bare key.

The kubeadm-emitted indent question you raised is now moot for correctness: the capture group preserves whatever indent exists (4 or 8 spaces both tested). Happy to test against a real kubelet.conf from the reporter if they can provide one.

[guoard]

LGTM.
Please update the title/description/release note though, they still describe the 8-space approach and the kubeadm 1.35 claim.

[guoard]

/lgtm

[mehrdadbn9]

@guoard thanks for the lgtm.

I'm preparing a Kubernetes GitHub org membership request (kubernetes/org, area/github-membership). Per the requirements I need two sponsors who are reviewers/approvers in an OWNERS file and have had close interactions with my work — your review here (kubespray-reviewers, per OWNERS_ALIASES) fits that.

Would you be willing to sponsor? If yes, a brief +1 on my membership issue once I open it is all that's needed. No pressure either way.

[guoard]

Would you be willing to sponsor?

Thanks, but I'll pass. I'm only a reviewer on kubespray and we've just crossed paths on a couple of PRs, not enough for me to vouch for someone.
The kubespray approvers know your work much better.

[VannTen]

@mehrdadbn9 I suggest you tone it down with the sponsoring requests. It might not be your intention (written communication is not always as easy as in person), but it comes across as a bit pushy, when you had few interactions with the people before.

[mehrdadbn9]

@guoard @VannTen Thank you both, genuinely — for the reviews and for telling me straight.

@guoard, understood completely, no hard feelings at all. Your lgtm and the backrefs catch already taught me a lot.

@VannTen, you're right and I appreciate you saying it. I don't know the community flow well yet and I pushed too hard on the sponsorship asks with too few interactions behind them. I'll stop the asks entirely and just keep doing the code work — starting with fixing the #13429 title/description you flagged.

If there's a better way I should be learning the process, I'm glad to hear it.