WSL2 rtnl deadlock after hv_netvsc sub-channel open fails with ENOMEM

[fiberproduct]

Windows Version

10.0.26200.8037

WSL Version

2.7.12.0

Are you using WSL 1 or WSL 2?

• WSL 2
• WSL 1

Kernel Version

6.18.33.2-2 (uname -r: 6.18.33.2-microsoft-standard-WSL2)

Distro Version

Ubuntu 24.04.4 LTS

Other Software

• .wslconfig at time of incident: memory=2560MB, processors=4, swap=2GB,
  autoMemoryReclaim=gradual (host has 8 GB RAM; limit since raised to 4096MB)
• Inside the VM: long-lived opencode CLI (Bun runtime) session ~735 MB RSS
  plus several ~150 MB child processes — the source of memory pressure

Repro Steps

Cannot reproduce on demand — it requires kernel memory exhaustion inside a
small VM. Single occurrence on 2026-08-28 ~05:30 UTC with full capture below.

Trigger profile:

1. Set .wslconfig memory cap well below steady-state process usage
   (2560 MB cap vs ~1.3 GB of resident processes + page cache).
2. Let memory pressure reach kernel OOM conditions (dmesg shows allocation
   failure reports from vmbus channel-open work).
3. From that moment, every process that calls getifaddrs() at startup
   (Node/Bun runtimes, ip addr, ...) hangs forever in D state — each
   retry adds another unkillable process, worsening the exhaustion.

Expected Behavior

A failed sub-channel open (ENOMEM) should unwind, release rtnl, and let the
interface keep working single-channel. Netlink queries must not be able to
wedge the whole control plane permanently, and the VM must remain
shut-downable (wsl --shutdown should not hang).

Actual Behavior

Under kernel memory pressure, the network stack wedged permanently:

1. Every process calling getifaddrs() hung forever in D state,
   wchan = rtnl_dumpit; SIGKILL undeliverable.
2. One child became an unreapable zombie.
3. wsl --shutdown itself hung indefinitely (graceful shutdown waits for
   processes to exit).
4. Only recovery: elevated taskkill /F on vmmemWSL.exe and wslservice.exe
   (service stuck in STOP_PENDING), then restart vmcompute + WSLService.

Notably, established connections / data-plane traffic kept working — only
netlink control/query paths hung, consistent with rtnl_mutex being held
forever while data paths use RCU.

Analysis: sub-channel open (netvsc_sc_open → vmbus_open) failed with -ENOMEM
while rtnl was held, and the error path appears to never release rtnl_mutex.
From that point every RTM_GETLINK netlink dump blocks forever — exactly where
the D-state processes were stuck (rtnl_dumpit).

Diagnostic Logs

Captured from dmesg while the wedge was live (the VM was later force-killed,
so this capture is the primary evidence):

hv_netvsc de1108d5-...  eth0: sub channel open failed: -12
hv_netvsc de1108d5-...  eth0: sub channel open failed: -12

with kernel stack trace:

__vmbus_open → vmbus_open → netvsc_sc_open → vmbus_add_channel_work
? rtnl_is_locked+0x19/0x30

followed by a full kernel allocation-failure report (Mem-Info dump) and:

WSL (107) ERROR: CheckConnection: getaddrinfo() failed: -5
warn_alloc: 1 callbacks suppressed

Related:

• WSL 2 Stalls During Boot: hv_balloon Memory Allocation Failure #12764 (hv_balloon memory allocation failure stalls WSL2) — closest analog
• Permanent RCU-tasks grace-period stall after wsl -t wedges all BPF trampoline attach/detach until wsl --shutdown #41136 (RCU-tasks stall wedges all BPF ops until wsl --shutdown) — same
  "irrecoverable kernel wait wedges a whole subsystem" shape, different
  subsystem; possibly a class of WSL2 kernel liveness bugs
• WSL 2 consumes massive amounts of RAM and doesn't return it #4166 (memory pressure background; does not cover the deadlock)
• KB5068861 breaks mirror networking with VPN #13724 shows the same CheckConnection: getaddrinfo() failed: -5 line in
  a different context (mirrored networking + VPN) — unrelated root cause

[github-actions]

Logs are required for review from WSL team

If this a feature request, please reply with '/feature'. If this is a question, reply with '/question'.
Otherwise, please attach logs by following the instructions below, your issue will not be reviewed unless they are added. These logs will help us understand what is going on in your machine.

How to collect WSL logs

Download and execute collect-wsl-logs.ps1 in an administrative powershell prompt:

Invoke-WebRequest -UseBasicParsing "https://raw.githubusercontent.com/microsoft/WSL/master/diagnostics/collect-wsl-logs.ps1" -OutFile collect-wsl-logs.ps1
Set-ExecutionPolicy Bypass -Scope Process -Force
.\collect-wsl-logs.ps1

The script will output the path of the log file once done.

If this is a networking issue, please use .\collect-wsl-logs.ps1 -LogProfile networking instead, following the instructions in Collect WSL logs for networking issues

Once completed please upload the output files to this GitHub issue.

See Collect WSL logs (recommended method).

If you choose to email these logs instead of attaching them to the bug, please send them to wsl-gh-logs@microsoft.com with the GitHub issue number in the subject, and include a link to your GitHub issue comment in the message body, and reply with '/emailed-logs'.

[fiberproduct]

Logs attached (WslLogs-trimmed.tar.gz, 1.4 MB, 69 files).

Note: I excluded logs.etl (274 MB uncompressed) to fit GitHub's 25 MB
attachment limit — it is a fresh trace started by the collector at 19:20 UTC+8
today, i.e. from the already-recovered healthy system, not from the incident
window (~05:30 UTC). The incident itself was non-reproducible and the VM had
to be force-killed, so the dmesg capture quoted in the report is the primary
incident evidence. Happy to email the full untrimmed bundle if needed.

WslLogs-trimmed.tar.gz

[github-actions]

No logs.etl found in the archive. Make sure that you ran collect-wsl-logs.ps1 as administrator and that the logs.etl file is in the archive.

Diagnostic information

.wslconfig found
Detected appx version: 2.7.12.0
No logs.etl found in archive.
Error while parsing the logs. See action page for details