[QUESTION] Can icecc be used on Docker?

[ericriff]

Sorry if this is explained somewhere, I looked around and I couldn't find any info.

I currently have a working icecc setup, but it is based on native (aka non-docker) builds.

I'm trying to migrate my workflow to use Docker instead, specifically vscode's devcontainers.

I have icecc installed and running on my container and iceccmon can see it. Yet when I launch a build from within the container, no jobs are assigned to the slave PCs.

I was wondering if this is a supported workflow and if any specific setup is required.

Thanks,

Eric.

[ericriff]

Just in case, this is the complete setup described on the picture.

• Laptop 1: Ubuntu 20.04, has the icecc scheduler installed. It is running the docker container which I'm using to build, shows up on icecc-mon as the 1a7.. node. This container is based on Ubuntu 22.04.
• Laptop 2: Ubuntu 22.04. Has iceccd installed and runnign.
  Both laptops are directly connected to the router through ethernet.

[ericriff]

Well, as soon as I posted this I made some small progress. Turns out iceccd was not running as root on the container.
Now it looks better

sudo iceccd -vvv
[5944] 2025-02-18 18:52:40: ICECREAM daemon 1.3.1 starting up (nice level 5) 
[5944] 2025-02-18 18:52:40: 16 CPU(s) online on this server
[5944] 2025-02-18 18:52:40: allowing up to 16 active jobs
[5944] 2025-02-18 18:52:40: supported features: env_xz env_zstd
[5944] 2025-02-18 18:52:40: not detaching
[5944] 2025-02-18 18:52:40: already a process group leader
[5944] 2025-02-18 18:52:40: ignoring localhost lo for broadcast
[5944] 2025-02-18 18:52:40: broadcast eth0 172.17.255.255
[5944] 2025-02-18 18:52:40: Netnames:
[5944] 2025-02-18 18:52:40: ICECREAM
[5944] 2025-02-18 18:52:40: starting to listen on all interfaces
[5944] 2025-02-18 18:52:40: ignoring localhost lo for broadcast
[5944] 2025-02-18 18:52:40: broadcast eth0 172.17.255.255
[5944] 2025-02-18 18:52:40: scheduler not yet found/selected.
[5944] 2025-02-18 18:52:40: Suitable scheduler found at 172.17.0.1:8765 (version: 42)
[5944] 2025-02-18 18:52:40: scheduler not yet found/selected.
[5944] 2025-02-18 18:52:43: scheduler is on 172.17.0.1:8765 (net ICECREAM)
[5944] 2025-02-18 18:52:43: Connected to scheduler (I am known as 172.17.0.2)

And icecc mon now properly lists the 16 cores of my container. Just to make sure I killed the iceccd which was running natively on the laptop where the container is running.
But no jobs are being assigned to the slave laptop anyways.

[ericriff]

This is the scheduler log. Nothing relevant to my untrained eye

icecc-scheduler -vvv
[1124312] 2025-02-18 15:57:59: ICECREAM scheduler 1.3.1 starting up, port 8765
[1124312] 2025-02-18 15:57:59: scheduler ready
[1124312] 2025-02-18 15:57:59: ignoring localhost lo for broadcast
[1124312] 2025-02-18 15:57:59: broadcast enx381428ecadc1 192.168.0.255
[1124312] 2025-02-18 15:57:59: broadcast docker0 172.17.255.255
[1124312] 2025-02-18 15:57:59: broadcast lxdbr0 10.163.218.1
[1124312] 2025-02-18 15:57:59: ignoring localhost lo for broadcast
[1124312] 2025-02-18 15:57:59: broadcast enx381428ecadc1 192.168.0.255
[1124312] 2025-02-18 15:57:59: broadcast docker0 172.17.255.255
[1124312] 2025-02-18 15:57:59: broadcast lxdbr0 10.163.218.1
[1124312] 2025-02-18 15:57:59: Received scheduler announcement from 192.168.0.146:54693 (version 42, netname ICECREAM)
[1124312] 2025-02-18 15:57:59: Received scheduler announcement from 172.17.0.1:54693 (version 42, netname ICECREAM)
[1124312] 2025-02-18 15:57:59: Received scheduler announcement from 10.163.218.1:54693 (version 42, netname ICECREAM)
[1124312] 2025-02-18 15:58:00: broadcast from 192.168.0.146:52443 (version 42)
[1124312] 2025-02-18 15:58:00: broadcast from 172.17.0.1:52443 (version 42)
[1124312] 2025-02-18 15:58:00: broadcast from 10.163.218.1:52443 (version 42)
[1124312] 2025-02-18 15:58:02: accepted 192.168.0.146
[1124312] 2025-02-18 15:58:32: broadcast from 172.17.0.2:49940 (version 42)
[1124312] 2025-02-18 15:58:32: broadcast from 192.168.0.42:57233 (version 42)
[1124312] 2025-02-18 15:58:35: accepted 172.17.0.2
[1124312] 2025-02-18 15:58:35: login 1a7f8dd9ab28 protocol version: 42 features: env_xz env_zstd []
[1124312] 2025-02-18 15:58:39: accepted 192.168.0.42
[1124312] 2025-02-18 15:58:39: login eric-Precision-5550 protocol version: 42 features: env_xz env_zstd []

[adriaandegroot]

Yes, it can. Well, maybe it depends on whether you mean "farm out from a docker container" or "send jobs into a docker container". At work-work we have 5 desktop machines, all beefy. They run one or more docker containers and we do our development inside the docker container. Then when running a build (inside the container) the jobs get farmed out to the desktop machines.

I can look up what exactly our setup is, there; it wasn't especially exciting to set up. But, if you mean the other way around, where jobs get sent into a container, I don't know. I think we put some non-zero effort into avoiding that, because our containers are supposed to be short-lived workspaces.