Skip to content

Updates the Docker images so that users run under Flux #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
slabasan merged 2 commits into from
Jul 7, 2025
Merged

Updates the Docker images so that users run under Flux #5

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
dcbfb9b
Launches Jupyter under Flux to allow us to emulate multi-node
ilumsden Jun 18, 2025
c78b897
Moves some bash setup to the Caliper Dockerfile and removes the insta…
ilumsden Jul 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 3 additions & 9 deletions 2025-HPDC/docker/Dockerfile.benchpark
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,15 +37,9 @@ USER root
RUN . /opt/global_py_venv/bin/activate && \
python3 -m pip install -r ${HOME}/benchpark/requirements.txt

RUN echo 'export PATH=/usr/bin:$PATH' >> ${HOME}/.bashrc && \
echo '. /opt/global_py_venv/bin/activate' >> ${HOME}/.bashrc && \
echo 'export LD_LIBRARY_PATH=/usr/lib:/usr/lib64:$LD_LIBRARY_PATH' >> ${HOME}/.bashrc && \
echo 'export PATH=${HOME}/benchpark/bin:$PATH' >> ${HOME}/.bashrc

RUN echo 'export PATH=/usr/bin:$PATH' >> ${HOME}/.bash_profile && \
echo '. /opt/global_py_venv/bin/activate' >> ${HOME}/.bash_profile && \
echo 'export LD_LIBRARY_PATH=/usr/lib:/usr/lib64:$LD_LIBRARY_PATH' >> ${HOME}/.bash_profile && \
echo 'export PATH=${HOME}/benchpark/bin:$PATH' >> ${HOME}/.bash_profile
RUN echo 'export PATH=${HOME}/benchpark/bin:$PATH' >> ${HOME}/.bashrc

RUN echo 'export PATH=${HOME}/benchpark/bin:$PATH' >> ${HOME}/.bash_profile

RUN chmod -R 777 ~/ ${HOME}

Expand Down
28 changes: 23 additions & 5 deletions 2025-HPDC/docker/Dockerfile.caliper
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,12 @@
#
# SPDX-License-Identifier: Apache-2.0

FROM ubuntu:noble
# FROM ubuntu:noble
FROM fluxrm/flux-sched:noble

# ubuntu:noble added a new 'ubuntu' user in the container.
# Get rid of it!
RUN userdel -r ubuntu
# RUN userdel -r ubuntu

USER root

Expand All @@ -24,10 +25,13 @@ RUN apt-get update && \
python3-dev \
python3-pip \
python3-venv \
openmpi-bin \
openmpi-common \
libopenmpi-dev \
git \
# NOTE: the flux-sched image already pulls and builds MPICH 4.2.2
# WITHOUT PMIx support (this is important because PMIx is a pain, and
# requires extra setup with Flux).
# openmpi-bin \
# openmpi-common \
# libopenmpi-dev \
&& rm -rf /var/lib/apt/lists/*

SHELL [ "/bin/bash", "-c" ]
Expand Down Expand Up @@ -97,12 +101,26 @@ RUN adduser \
--force-badname \
${NB_USER}

# NOTE: this should NEVER be uncommented by the time we push to GHCR
RUN adduser ${NB_USER} sudo
RUN echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers

RUN chmod -R 777 ~/ ${HOME}

ENV SHELL=/usr/bin/bash

RUN mkdir -p ${HOME}/.local/share && \
chmod 777 ${HOME}/.local/share

RUN echo $(flux env)

RUN echo 'export PATH=/usr/bin:$PATH' >> ${HOME}/.bashrc && \
echo '. /opt/global_py_venv/bin/activate' >> ${HOME}/.bashrc && \
echo 'export LD_LIBRARY_PATH=/usr/lib:/usr/lib64:$LD_LIBRARY_PATH' >> ${HOME}/.bashrc

RUN echo 'export PATH=/usr/bin:$PATH' >> ${HOME}/.bash_profile && \
echo '. /opt/global_py_venv/bin/activate' >> ${HOME}/.bash_profile && \
echo 'export LD_LIBRARY_PATH=/usr/lib:/usr/lib64:$LD_LIBRARY_PATH' >> ${HOME}/.bash_profile

USER ${NB_USER}
WORKDIR ${HOME}
4 changes: 3 additions & 1 deletion 2025-HPDC/docker/Dockerfile.spawn
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,8 @@ RUN chmod 777 /entrypoint.sh
RUN chmod 777 /local-entrypoint.sh

USER ${NB_USER}
ENV SHELL=/usr/bin/bash
ENV FLUX_URI_RESOLVE_LOCAL=t

RUN git clone https://github.com/LLNL/thicket-tutorial.git ${HOME}/thicket-tutorial && \
cd ${HOME}/thicket-tutorial && \
Expand All @@ -51,4 +53,4 @@ RUN git clone https://github.com/LLNL/thicket-tutorial.git ${HOME}/thicket-tutor
EXPOSE 8888
ENTRYPOINT [ "tini", "--" ]

CMD ["jupyter", "lab"]
CMD ["flux", "start", "jupyter", "lab"]
6 changes: 5 additions & 1 deletion 2025-HPDC/docker/spawn-entrypoint.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,8 @@
#
# SPDX-License-Identifier: Apache-2.0

/opt/global_py_venv/bin/jupyterhub-singleuser
num_cores_per_node=2
total_num_cores=$(nproc --all)
num_brokers=$(( $total_num_cores / $num_cores_per_node ))

/usr/bin/mpiexec.hydra -n $num_brokers -bind-to core:$num_cores_per_node /usr/bin/flux start /opt/global_py_venv/bin/jupyterhub-singleuser
6 changes: 5 additions & 1 deletion 2025-HPDC/docker/spawn-local-entrypoint.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
#!/usr/bin/env bash

/opt/global_py_venv/bin/jupyter-lab --ip=0.0.0.0
num_cores_per_node=2
total_num_cores=$(nproc --all)
num_brokers=$(( $total_num_cores / $num_cores_per_node ))

/usr/bin/mpiexec.hydra -n $num_brokers -bind-to core:$num_cores_per_node /usr/bin/flux start /opt/global_py_venv/bin/jupyter-lab --ip=0.0.0.0