[Bug]: reaper reuse path resolves the Docker host through reaper_default even when bridge exists; another session's Ryuk then removes it (no route to host) #3941

Testcontainers version

v0.44.0 (the code is unchanged on main at 9d2885d)

Using the latest Testcontainers version?

Yes

Host OS

Linux (Ubuntu 24.04 for the reproduction below; first seen on Linux self-hosted CI runners)

Host arch

x86_64 for the reproduction; arm64 on the CI runners

Go version

1.26.4 (golang:1.26 image) for the reproduction; 1.26.6 on CI

Docker version

Docker Engine 29.5.3, API 1.54

Docker info

Server Version: 29.5.3
Operating System: Ubuntu 24.04.4 LTS
OSType: linux
Architecture: x86_64
Storage Driver: overlayfs
Cgroup Driver: systemd

What happened?

When tests run inside a container (Docker-out-of-Docker, /var/run/docker.sock mounted) and one go test runs more than one package, the later test processes of a session resolve the reaper's address through a network named reaper_default, although the default bridge network exists.

reaper_default is created with the creating session's labels (org.testcontainers.sessionId, org.testcontainers.reap=true). When that session ends, its Ryuk removes the network. Any other session on the same Docker daemon that reached its reaper through reaper_default's gateway then fails to create containers:

create container: reaper: connect: dial reaper 172.19.0.1:45201: dial tcp 172.19.0.1:45201: connect: no route to host

On CI, where several runner containers share one Docker daemon, this showed up as intermittent failures. A container creation failed on the reaper connect ("no route to host" or a timeout), or the connect retries used up the test's deadline.

Why

1. The first process of a session creates Ryuk through a provider built with WithDefaultBridgeNetwork(Bridge) (provider.go#L114). Inside a container, DaemonHost() then resolves to the bridge gateway.
2. A later process of the same session reuses that Ryuk: reuseOrCreate → lookupContainer (reaper.go#L158). lookupContainer builds its own provider with NewDockerProvider() without WithDefaultBridgeNetwork (reaper.go#L165).
3. fromContainer computes the endpoint with that provider (reaper.go#L357): Host() → daemonHostLocked() → InAContainer() (docker.go#L1636) → ensureDefaultNetworkLocked() (docker.go#L1773).
4. With an empty defaultBridgeNetworkName, case p.defaultBridgeNetworkName (docker.go#L1789) never matches bridge. So the provider uses an existing reaper_default, or creates one (docker.go#L1802), with GenericLabels() (docker.go#L1805): the creating session's ID and reap=true. As far as I can tell, Issue #243: Introducing the default network in case "bridge" is disabled #244 introduced reaper_default for environments where bridge is unavailable, not for this case.
5. When the creating session ends, its Ryuk prunes reaper_default. A concurrent session whose reused reaper resolved to that network's gateway can no longer reach its own Ryuk.

Minimal reproduction

The module depends only on github.com/testcontainers/testcontainers-go v0.44.0. Packages a, b and c each contain this test:

func TestStartsOneContainer(t *testing.T) {
	c, err := testcontainers.GenericContainer(context.Background(), testcontainers.GenericContainerRequest{
		ContainerRequest: testcontainers.ContainerRequest{
			Image:        "redis:7-alpine",
			ExposedPorts: []string{"6379/tcp"},
			WaitingFor:   wait.ForListeningPort("6379/tcp"),
		},
		Started: true,
	})
	testcontainers.CleanupContainer(t, c)
	if err != nil {
		t.Fatal(err)
	}
}

Package d starts one such container, sleeps 30 s, then starts another.

docker network create tcrepro
RUN="docker run --rm --network tcrepro -v /var/run/docker.sock:/var/run/docker.sock -v $PWD:/src -w /src golang:1.26"
$RUN go test -count=1 -p 1 ./a ./b        # session A
$RUN go test -count=1 -p 1 -v ./c ./d     # session B, started right after A ends

What I see:

• During session A. When b logs 🔥 Reaper obtained from Docker for this test session, reaper_default appears (172.19.0.0/16, gateway 172.19.0.1), labelled org.testcontainers.sessionId=<session A> and org.testcontainers.reap=true. It is removed about 10 s after A ends.
• During session B. d reuses B's Ryuk while A's reaper_default still exists, so its reaper endpoint becomes 172.19.0.1:<port>. After A's Ryuk removes the network, d's second container fails with the error above.
• With the workaround. Setting TESTCONTAINERS_HOST_OVERRIDE to the bridge gateway (172.17.0.1 here) for session B makes the same sequence pass.

-p 1 only makes the timing deterministic. The reuse path is taken whenever a later process of the session finds that session's Ryuk already running.

Relevant log output

# session B (-v), trimmed
  Test SessionID: 54c8c31a0bc4679563710b6f7ae11b120b8fb368ce41c1f79c4c0473bcdfccef
--- PASS: TestStartsOneContainer (1.74s)
  Test SessionID: 54c8c31a0bc4679563710b6f7ae11b120b8fb368ce41c1f79c4c0473bcdfccef
2026/10/07 05:45:16 🔥 Reaper obtained from Docker for this test session 73c48704
    d_test.go:32: create container: reaper: connect: dial reaper 172.19.0.1:45201: dial tcp 172.19.0.1:45201: connect: no route to host
--- FAIL: TestStartsTwoContainersApart (41.55s)

Additional information

Suggested fix. Build the provider in lookupContainer with WithDefaultBridgeNetwork(Bridge), or reuse the provider passed into reuseOrCreate, so that the create path and the reuse path resolve the same host. Separately, a network that every session on the daemon may resolve through probably should not carry one session's reap labels.

Workaround. When running inside a container, set TESTCONTAINERS_HOST_OVERRIDE to the default bridge gateway before the first container is created.

Related:

• [Bug]: Cannot connect to ryuk when running test inside docker container #1373 (closed): the analysis there covers the newReaper path, not the reuse path;
• [Bug]: Parallel reapers creating error #3242: multi-package reaper reuse races;
• [Bug]: reaper reuse finds an exited ryuk container and times out instead of creating a new one #3867 / fix: recreate reaper when the existing container is not running #3868: another reuse-path problem; fix: recreate reaper when the existing container is not running #3868 leaves NewDockerProvider() in lookupContainer unchanged;
• Issue #243: Introducing the default network in case "bridge" is disabled #244: introduced reaper_default.