[BUG] Containers do not start after migrating to a new host

[legomind]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

After copying my data to the new server, kasm_prox is not reachable to any outside container.

Expected Behavior

Containers should start up properly

Steps To Reproduce

1. Setup a kasm install using README.
2. stop the kasm container
3. either copy data and docker-compose.yaml to a new server or uninstall docker, and reinstall.
4. run docker compose up -d on new server or fresh docker installation

Environment

OS: Rocky Linux 10
Install docker using official documentation for centos

CPU architecture

x86-64

Docker creation

kasm:
    image: lscr.io/linuxserver/kasm:latest
    container_name: kasm
    privileged: true
    security_opt:
      - apparmor:rootlesskit
    environment:
      - KASM_PORT=8443
      - DOCKER_MTU=1500 #optional
    volumes:
      - ./kasm/data:/opt
      - ./kasm/profiles:/profiles
    ports:
      - 3000:3000
      - 8443:8443
    restart: unless-stopped

Container logs

logs after migration, including several restarts:


kasm  | [migrations] started
kasm  | [migrations] no migrations found
kasm  | usermod: no changes
kasm  | ───────────────────────────────────────
kasm  |
kasm  |       ██╗     ███████╗██╗ ██████╗
kasm  |       ██║     ██╔════╝██║██╔═══██╗
kasm  |       ██║     ███████╗██║██║   ██║
kasm  |       ██║     ╚════██║██║██║   ██║
kasm  |       ███████╗███████║██║╚██████╔╝
kasm  |       ╚══════╝╚══════╝╚═╝ ╚═════╝
kasm  |
kasm  |    Brought to you by linuxserver.io
kasm  | ───────────────────────────────────────
kasm  |
kasm  | To support LSIO projects visit:
kasm  | https://www.linuxserver.io/donate/
kasm  |
kasm  | ───────────────────────────────────────
kasm  | GID/UID
kasm  | ───────────────────────────────────────
kasm  |
kasm  | User UID:    911
kasm  | User GID:    911
kasm  | ───────────────────────────────────────
kasm  | [custom-init] No custom files found, skipping...
kasm  | [ls.io-init] done.
kasm  | time="2025-09-24T00:05:24Z" level=info msg=" * Serving Flask app 'plugin'" plugin=8d74e14ff16e91ea3bbdbc7ca077f8525d796bdf3f9fb06b120fa82b00026202
kasm  | time="2025-09-24T00:05:24Z" level=info msg=" * Debug mode: on" plugin=8d74e14ff16e91ea3bbdbc7ca077f8525d796bdf3f9fb06b120fa82b00026202
kasm  | time="2025-09-24T00:05:44.120696461Z" level=error msg="failed to enable plugin" error="dial unix /run/docker/plugins/8d74e14ff16e91ea3bbdbc7ca077f8525d796bdf3f9fb06b120fa82b00026202/kasmnetwork.sock: connect: no such file or directory" id=8d74e14ff16e91ea3bbdbc7ca077f8525d796bdf3f9fb06b120fa82b00026202
kasm  | time="2025-09-24T00:06:56.880517267Z" level=error msg="779ede6e5f66c564d128bc53aa4bcfea9373e9614377764937de902d419dc906 cleanup: failed to delete container from containerd: NotFound: container \"779ede6e5f66c564d128bc53aa4bcfea9373e9614377764937de902d419dc906\" in namespace \"moby\": not found"
kasm  | time="2025-09-24T00:07:00.204300962Z" level=error msg="failed to start container" container=779ede6e5f66c564d128bc53aa4bcfea9373e9614377764937de902d419dc906 error="failed to set up container networking: failed to add endpoint: plugin kasmweb/sidecar:1.2 found but disabled"
kasm  | [migrations] started
kasm  | [migrations] no migrations found
kasm  | usermod: no changes
kasm  | ───────────────────────────────────────
kasm  |
kasm  |       ██╗     ███████╗██╗ ██████╗
kasm  |       ██║     ██╔════╝██║██╔═══██╗
kasm  |       ██║     ███████╗██║██║   ██║
kasm  |       ██║     ╚════██║██║██║   ██║
kasm  |       ███████╗███████║██║╚██████╔╝
kasm  |       ╚══════╝╚══════╝╚═╝ ╚═════╝
kasm  |
kasm  |    Brought to you by linuxserver.io
kasm  | ───────────────────────────────────────
kasm  |
kasm  | To support LSIO projects visit:
kasm  | https://www.linuxserver.io/donate/
kasm  |
kasm  | ───────────────────────────────────────
kasm  | GID/UID
kasm  | ───────────────────────────────────────
kasm  |
kasm  | User UID:    911
kasm  | User GID:    911
kasm  | ───────────────────────────────────────
[custom-init] No custom files found, skipping...
kasm  | [ls.io-init] done.
kasm  | time="2025-09-24T13:01:18.246547078Z" level=error msg="copy shim log after reload" error="read /proc/self/fd/6: file already closed" namespace=moby
kasm  | time="2025-09-24T13:01:18.372836678Z" level=error msg="copy shim log after reload" error="read /proc/self/fd/6: file already closed" namespace=moby
kasm  | time="2025-09-24T13:01:18.488947660Z" level=error msg="copy shim log after reload" error="read /proc/self/fd/6: file already closed" namespace=moby
kasm  | time="2025-09-24T13:01:18.606001740Z" level=error msg="copy shim log after reload" error="read /proc/self/fd/6: file already closed" namespace=moby
kasm  | time="2025-09-24T13:01:23Z" level=info msg=" * Serving Flask app 'plugin'" plugin=8d74e14ff16e91ea3bbdbc7ca077f8525d796bdf3f9fb06b120fa82b00026202
kasm  | time="2025-09-24T13:01:23Z" level=info msg=" * Debug mode: on" plugin=8d74e14ff16e91ea3bbdbc7ca077f8525d796bdf3f9fb06b120fa82b00026202
kasm  | [migrations] started
kasm  | [migrations] no migrations found
kasm  | usermod: no changes
kasm  | ───────────────────────────────────────
kasm  |
kasm  |       ██╗     ███████╗██╗ ██████╗
kasm  |       ██║     ██╔════╝██║██╔═══██╗
kasm  |       ██║     ███████╗██║██║   ██║
kasm  |       ██║     ╚════██║██║██║   ██║
kasm  |       ███████╗███████║██║╚██████╔╝
kasm  |       ╚══════╝╚══════╝╚═╝ ╚═════╝
kasm  |
kasm  |    Brought to you by linuxserver.io
kasm  | ───────────────────────────────────────
kasm  |
kasm  | To support LSIO projects visit:
kasm  | https://www.linuxserver.io/donate/
kasm  |
kasm  | ───────────────────────────────────────
kasm  | GID/UID
kasm  | ───────────────────────────────────────
kasm  |
kasm  | User UID:    911
kasm  | User GID:    911
kasm  | ───────────────────────────────────────
kasm  | [custom-init] No custom files found, skipping...
kasm  | [ls.io-init] done.
kasm  | time="2025-09-24T13:09:15.595710627Z" level=error msg="copy shim log after reload" error="read /proc/self/fd/6: file already closed" namespace=moby
kasm  | time="2025-09-24T13:09:15.704826580Z" level=error msg="copy shim log after reload" error="read /proc/self/fd/6: file already closed" namespace=moby
kasm  | time="2025-09-24T13:09:15.813538766Z" level=error msg="copy shim log after reload" error="read /proc/self/fd/6: file already closed" namespace=moby
kasm  | time="2025-09-24T13:09:15.922570259Z" level=error msg="copy shim log after reload" error="read /proc/self/fd/6: file already closed" namespace=moby
kasm  | time="2025-09-24T13:09:16.032831457Z" level=error msg="copy shim log after reload" error="read /proc/self/fd/6: file already closed" namespace=plugins.moby
kasm  | time="2025-09-24T13:09:16Z" level=info msg=" * Serving Flask app 'plugin'" plugin=8d74e14ff16e91ea3bbdbc7ca077f8525d796bdf3f9fb06b120fa82b00026202
kasm  | time="2025-09-24T13:09:16Z" level=info msg=" * Debug mode: on" plugin=8d74e14ff16e91ea3bbdbc7ca077f8525d796bdf3f9fb06b120fa82b00026202
kasm  | time="2025-09-24T13:11:01.568077452Z" level=error msg="copy stream failed" error="reading from a closed fifo" stream=stdout
kasm  | time="2025-09-24T13:11:01.569285122Z" level=error msg="Error running exec db1f09f7cda93742f33cdce79b2e110905bbc13da00387413684cdc54cfd6ac4 in container: OCI runtime exec failed: exec failed: unable to start container process: exec: \"bash\": executable file not found in $PATH: unknown"

[github-actions]

Thanks for opening your first issue here! Be sure to follow the relevant issue templates, or risk having this issue marked as invalid.

[legomind]

Ok, so it it turns out the problem is overlapping network ranges. after the migration to the new host the internal docker network inside the kasm container had overlapping ips with one of my outer docker networks. I resolved the immediate problem by deleting the internal kasm networks, mounting the below json to /etc/docker/daemon.json, and then recreating the kasm networks.

At the very least, this should be put in the README

Inside kasm container: (docker exec -it kasm bash)

cd /op/kasm/current/docker
export KASM_UID=$(id kasm -u)
export KASM_GID=$(id kasm -g)
docker compose down
docker network rm kasm_default_network
docker network rm kasm_sidecar_network

kasm/docker_daemon.json

{
    "runtimes": {
        "nvidia": {
            "args": [],
            "path": "nvidia-container-runtime"
        }
    },
 "default-address-pools": [
    {
      "base": "172.30.0.0/16",
      "size": 24
    }
  ]
}

docker-compose.yaml

kasm:
    image: lscr.io/linuxserver/kasm:latest
    container_name: kasm
    privileged: true
    security_opt:
      - apparmor:rootlesskit #optional
    environment:
      - KASM_PORT=8443
      - DOCKER_MTU=1500 #optional
    volumes:
      - ./kasm/data:/opt
      - ./kasm/profiles:/profiles
      - ./kasm/docker_daemon.json:/etc/docker/daemon.json
    ports:
      - 3000:3000
      - 8443:8443
    restart: unless-stopped

on host

docker compose up -d

Inside kasm container: (docker exec -it kasm bash)

docker network create kasm_default_network
docker network create kasm_sidecar_network
/opt/kasm/bin/start

[ConsularParadi]

Facing similar issues, even though mine was a fresh installation

time="2025-10-02T00:33:03.509769221Z" level=error msg="copy shim log after reload" error="read /proc/self/fd/6: file already closed" namespace=moby
time="2025-10-02T00:33:03.654293179Z" level=error msg="copy shim log after reload" error="read /proc/self/fd/6: file already closed" namespace=moby
time="2025-10-02T00:33:03.801187280Z" level=error msg="copy shim log after reload" error="read /proc/self/fd/6: file already closed" namespace=moby
time="2025-10-02T00:33:04.094794125Z" level=error msg="copy shim log after reload" error="read /proc/self/fd/6: file already closed" namespace=moby
time="2025-10-02T00:33:04.235214775Z" level=error msg="cleanup working directory in namespace" error="open /opt/docker/containerd/daemon/io.containerd.runtime.v2.task/moby: no such file or directory" namespace=moby
time="2025-10-02T00:33:04.236606859Z" level=error msg="copy shim log after reload" error="read /proc/self/fd/6: file already closed" namespace=plugins.moby
time="2025-10-02T00:33:04.366891513Z" level=error msg="cleanup working directory in namespace" error="open /opt/docker/containerd/daemon/io.containerd.runtime.v2.task/plugins.moby: no such file or directory" namespace=plugins.moby
time="2025-10-02T00:38:36Z" level=info msg=" * Serving Flask app 'plugin'" plugin=fa66cd3ed9a924bf1321fa1c372506a5080da476d765245a0e26adc77ba96442
time="2025-10-02T00:38:36Z" level=info msg=" * Debug mode: on" plugin=fa66cd3ed9a924bf1321fa1c372506a5080da476d765245a0e26adc77ba96442
time="2025-10-02T00:44:03.307907933Z" level=error msg="copy shim log" error="read /proc/self/fd/56: file already closed" namespace=moby
time="2025-10-02T00:44:03.308792873Z" level=error msg="copy stream failed" error="reading from a closed fifo" stream=stdout
time="2025-10-02T00:44:03.308794066Z" level=error msg="copy stream failed" error="reading from a closed fifo" stream=stderr
time="2025-10-02T00:53:11.556655121Z" level=error msg="copy stream failed" error="reading from a closed fifo" stream=stdout
time="2025-10-02T00:53:11.556676342Z" level=error msg="copy stream failed" error="reading from a closed fifo" stream=stderr
time="2025-10-02T00:57:09.861210664Z" level=error msg="copy shim log" error="read /proc/self/fd/56: file already closed" namespace=moby
time="2025-10-02T00:57:09.862121452Z" level=error msg="copy stream failed" error="reading from a closed fifo" stream=stderr
time="2025-10-02T00:57:09.862121981Z" level=error msg="copy stream failed" error="reading from a closed fifo" stream=stdout

The above mentioned solution did not seem to work.