kernel: ipv6.disable=0 with no IPv6 routing causes Happy Eyeballs latency on all outbound sandbox connections #3585

Symptom

Every outbound connection from a sandbox to an external host that has AAAA records takes 250 ms or more longer than expected. The extra latency appears on the first TCP connection per destination — pip install, npm install, HTTP API calls, curl, etc.

# inside sandbox
time curl -s https://pypi.org -o /dev/null
# real    0m0.582s   (should be ~0.1s from same region)

The delay is consistent and reproducible; once the TCP connection is established, transfer speed is normal.

Root cause

packages/orchestrator/pkg/sandbox/fc/kernel_args.go:96-97 explicitly enables IPv6 inside every Firecracker guest:

"ipv6.disable":  "0",   // IPv6 enabled
"ipv6.autoconf": "1",   // SLAAC enabled

However packages/orchestrator/pkg/sandbox/fc/process.go:378 configures the guest network with an IPv4-only ip= kernel parameter:

// IPv4 configuration - format: [local_ip]::[gateway_ip]:[netmask]:hostname:iface:dhcp_option:[dns]
ipv4 := fmt.Sprintf("%s::%s:%s:instance:%s:off:%s",
    p.slot.NamespaceIP(), p.slot.TapIPString(), p.slot.TapMaskString(),
    p.slot.VpeerName(), p.slot.TapName())

The host side of the tap device only configures IPv4 NAT rules (network.go:265); there is no IPv6 router advertisement (radvd) or IPv6 default route on the host tap interface.

Result:

• IPv6 is active in the guest kernel
• SLAAC runs on the tap interface but finds no IPv6 router → only a link-local fe80:: address is obtained
• No global unicast IPv6 address, no default IPv6 route
• All IPv6 traffic is unroutable, but the protocol stack is fully live

When the guest connects to any dual-stack host, the Linux kernel's address selection (RFC 6724) prefers IPv6:

connect("api.openai.com:443")
  → DNS: returns both A + AAAA
  → kernel tries IPv6 AAAA address first
  → no route → EHOSTUNREACH / NDP solicitation times out
  → fallback to IPv4 A record  (~250 ms per RFC 8305)
  → connection finally succeeds
  → every first-connection pays 250 ms+

The /etc/sysctl.conf written by packages/orchestrator/pkg/template/build/phases/base/provision.sh also has no IPv6 disable entry — only fs.inotify.max_user_watches and vm.compaction_proactiveness are set — so the guest OS never suppresses IPv6 at the sysctl layer either.

Why it matters

• Every sandbox is affected, regardless of workload
• Any pip, npm, cargo, apt fetch that hits a dual-stack host pays the penalty per connection
• Latency-sensitive API calls (OpenAI, GitHub, GCP, AWS) all have AAAA records
• The penalty compounds: a package install that opens 20 TCP connections silently loses 5+ seconds

Proposed fix

Change kernel_args.go:96 to disable IPv6 until the networking layer supports a complete IPv6 stack:

// Before
"ipv6.disable":  "0",
"ipv6.autoconf": "1",

// After
"ipv6.disable": "1",
// ipv6.autoconf is redundant when disable=1, remove it

This is a single-line kernel cmdline change with no other code impact. If full IPv6 support is planned in the future, the correct order is: first configure IPv6 routing on the host tap (RA, prefix delegation, ip6tables), then set ipv6.disable=0.