[Bug]: Testcontainers fails with Docker 29.0.1 - overlay mount errors

[tamirault]

[Bug]: Testcontainers fails with Docker 29.0.1 - overlay mount errors

Testcontainers version

4.4.0 (Testcontainers.Redis)

.NET version

.NET 9.0

Using Microsoft Visual Studio?

No

Using JetBrains Rider?

Yes

Environment

• Docker version: 29.0.1
• containerd version: v2.1.5
• Platform: Running inside Dagger pipeline (docker-in-docker scenario)
• OS: Ubuntu 22.04 (GitHub Actions runner)

Bug description

When running tests that use Testcontainers with Docker 29.0.1, container creation fails with overlay filesystem mount errors.

Error:

OneTimeSetUp: Docker.DotNet.DockerApiException : Docker API responded with status code=InternalServerError, 
response={"message":"failed to mount /tmp/containerd-mount: mount source: \"overlay\", 
target: \"/tmp/containerd-mount\", fstype: overlay, flags: 0, 
data: \"workdir=/var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.overlayfs/snapshots/9/work,...
err: invalid argument"}

Steps to reproduce

1. Use Testcontainers.Redis 4.4.0 in .NET 9 tests
2. Run tests in environment with Docker 29.0.1 (e.g., docker:29-dind image)
3. Container creation fails during OneTimeSetUp when starting Redis container

Test code:

[OneTimeSetUp]
public async Task GlobalSetup()
{
    _redisContainer = new RedisBuilder().Build();
    await _redisContainer!.StartAsync(); // Fails here
}

Expected behaviour

Tests should work with Docker 29.0.1, as it's the current stable Docker release.

Actual behaviour

Container creation fails with overlay mount errors. Tests pass when using Docker 28.x.

Workaround

Pin Docker to version 28:

• In Dagger: dag.docker().daemon().withVersion("28")
• Or use docker:28-dind image instead of latest

Stack trace

at Docker.DotNet.DockerClient.HandleIfErrorResponseAsync(...) in /_/src/Docker.DotNet/DockerClient.cs:line 492
at Docker.DotNet.DockerClient.MakeRequestAsync[...](...) in /_/src/Docker.DotNet/DockerClient.cs:line 257
at Docker.DotNet.DockerClient.MakeRequestAsync[...](...) in /_/src/Docker.DotNet/DockerClient.cs:line 232
at Docker.DotNet.ContainerOperations.CreateContainerAsync(...) in /_/src/Docker.DotNet/Endpoints/ContainerOperations.cs:line 64
at Testcontainers.Clients.DockerContainerOperations.RunAsync(...) in /_/src/Testcontainers/Clients/DockerContainerOperations.cs:line 225

Additional information

• Issue started appearing when Docker 29 was released
• Related to containerd v2.1.5 + Docker 29 overlay filesystem changes
• Affects docker-in-docker scenarios (Dagger, CI environments)

Related issues

• Dagger discussion: Dagger fails with containerd v2 on GitHub Actions ubuntu-22.04 (testcontainers overlay mount error) dagger/dagger#11427

[HofmeisterAn]

Yes, the current version of Docker.DotNet (the library Testcontainers depends on) isn't compatible with Docker Engine v29.

I've added a feature that lets you downgrade the Docker Engine API (#1576). It pins the API version to 1.44. Testcontainers for Java does something similar, though I'm not sure whether it cause other issues.

For now, there are two workarounds:

• Don't use Docker Engine v29 (pin it to v28), as you already did.
• Downgrade the Docker Engine API (available with the next release).

Once I've fully recovered from my cold, I'll continue working on proper v29 support.

I'll close the issue for now since there's nothing more to add.