Adds support and instructions for local testing of the tutorial

.gitignore

@@ -1,3 +1,6 @@
+**/.jupyter/
+**/.local/
+
 # Byte-compiled / optimized / DLL files
 __pycache__/
 *.py[cod]

2025-HPDC/README.rst

@@ -4,22 +4,26 @@ HPDC 2025 Tutorial
 
 This directory contains the materials for the HPDC 2025 tutorial. The following subsections go over the contains of the material.
 
--------------
+--------
+Contents
+--------
+
+^^^^^^^^^^^^^
 Tutorial Code
--------------
+^^^^^^^^^^^^^
 
 The code elements of this tutorial (e.g., Jupyter notebooks, command-line scripts, Markdown/RST instruction files) can all be found in the :code:`tutorial-code` subdirectory. If materials are actually stored in other git repositories, they can be accessed from this subdirectory
 via a git submodule.
 
-------
+^^^^^^
 Slides
-------
+^^^^^^
 
 The slides used in presenting this tutorial can be found in the :code:`slides` subdirectory.
 
-------
+^^^^^^
 Docker
-------
+^^^^^^
 
 The Docker definition files (i.e., Dockerfiles) for all the necessary containers can be found in the :code:`docker` subdirectory. There are currently 5 definition files:
 
@@ -29,9 +33,9 @@ The Docker definition files (i.e., Dockerfiles) for all the necessary containers
 4. :code:`Dockerfile.spawn`: download tutorial materials, download any remaining necessary packages, and do other setup work on top of the image produced by :code:`Dockerfile.benchpark`
 5. :code:`Dockerfile.init`: ensure user permissions are correct using the super-minimal :code:`alpine/git` image from DockerHub
 
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+"""""""""""""""""""""""""""""""""""""""
 Testing the Builds of the Docker Images
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+"""""""""""""""""""""""""""""""""""""""
 
 To enable automated testing of the Docker images, all edits to the Dockerfiles above should be done in a branch with an open PR. When a PR is open, a GitHub Actions CI will
 run and ensure that the images can be built. To properly configure the CI, edit the :code:`github_ci_matrix.json` file in the root of this repository as follows:
@@ -41,17 +45,17 @@ run and ensure that the images can be built. To properly configure the CI, edit
 
 The CI reads :code:`github_ci_matrix.json` to get values shared by the matrices of all GitHub Actions jobs.
 
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
 Pushing the Docker Images to GitHub Container Registry (GHCR)
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
 
 Before trying to push to GHCR, someone with the necessary permissions should make sure this repo can push to these images in GHCR (**change names when we decide on appropriate ones**):
 
-* ghcr.io/LLNL/caliper
-* ghcr.io/LLNL/thicket
-* ghcr.io/LLNL/benchpark
-* ghcr.io/LLNL/reproducible-benchmarking-spawn
-* ghcr.io/LLNL/reproducible-benchmarking-init
+* ghcr.io/llnl/caliper
+* ghcr.io/llnl/thicket
+* ghcr.io/llnl/benchpark
+* ghcr.io/llnl/reproducible-benchmarking-spawn
+* ghcr.io/llnl/reproducible-benchmarking-init
 
 If these images do not yet exist, your first push will properly set the permissions. If these images do exist, follow the instructions
 `here <https://docs.github.com/en/packages/learn-github-packages/configuring-a-packages-access-control-and-visibility#ensuring-workflow-access-to-your-package>`_
@@ -66,10 +70,70 @@ After ensuring this repository has the necessary permissions, to push the Docker
 5. In the popup menu that appears, select the "main" branch and fill out the requested information
 6. Click the green "Run workflow" button to start the process and building and pushing images
 
---------------
+^^^^^^^^^^^^^^
 Infrastructure
---------------
+^^^^^^^^^^^^^^
 
 All the infrastructure needed to deploy the tutorial to a Kubernetes cluster with JupyterHub is contained in the :code:`infrastructure` subdirectory.
 This infrastructure is generated by the tool `here <https://lc.llnl.gov/gitlab/lumsden1/hpcic-k8s-configurer>`_.
 The infrastructure can be regenerated as-is using :code:`infrastructure/config.toml`.
+
+----------------------------
+Testing the Tutorial Locally
+----------------------------
+
+To test the tutorial locally, you first need to build all the Docker images except the init image. Before building,
+keep in mind the following dependencies between images:
+
+.. code-block::
+
+   ghcr.io/llnl/caliper --> ghcr.io/llnl/thicket --> ghcr.io/llnl/benchpark --> ghcr.io/llnl/reproducible-benchmarking-spawn
+
+Because of these dependencies, the first thing you should figure out is which (if any) images you need to build locally.
+If a Dockerfile has changes that are **not** on GHCR, you will need to build that image *and all downstream images (based on the flowchart above)*
+locally before testing. To build an image locally, run the following from this directory (**not the** :code:`docker` **directory**):
+
+.. code-block:: bash
+
+   $ docker build -t <image_name> -f ./docker/<dockerfile_for_image> . # Note the trailing "." 
+
+In the command above, :code:`<image_name>` should be one of the GHCR URLs above, followed by a colon, followed by a tag. It could look something
+like :code:`ghcr.io/llnl/benchpark:hpdc-2025`. Note that :code:`<iamge_name>` **must** match the value of the :code:`FROM` directive
+for the dependent image. For example, to get the :code:`<image_name>` field for :code:`ghcr.io/llnl/benchpark`, look for the :code:`FROM` directive
+in :code:`./docker/Dockerfile.spawn`.
+
+If all the changes to the corresponding Dockerfiles in :code:`docker` have already been pushed to GHCR, you do not need to build locally.
+Instead, you should just pull the spawn image using:
+
+.. code-block:: bash
+
+   $ docker pull ghcr.io/llnl/reproducible-benchmarking-spawn:<tag>
+
+You should replace :code:`<tag>` in the command above with the GHCR tag of the image you want to pull.
+
+After you have a built spawn image (either by building locally or by pulling from GHCR), you can run the spawn image locally
+by running the following command:
+
+.. code-block:: bash
+
+   $ docker run --rm -it --entrypoint <entrypoint> --name reproducible_benchmark_tutorial_local -p 8888:8888 <spawn_image_name>
+
+In the command above, :code:`<spawn_image_name>` is the name of the built spawn image. If you built that image locally, this argument
+should match the value you passed to the :code:`-t` flag of :code:`docker build` when building the spawn image. If you pulled the image
+from GHCR, this argument should be :code:`ghcr.io/llnl/reproducible-benchmarking-spawn:<tag>`.
+
+The :code:`<entrypoint>` field in the command above dictates what command runs within the container immediately after startup.
+It can be one of three values:
+
+1. :code:`/local-entrypoint.sh`: this entrypoint script will start a JupyterLab instance and make it available from outside the container.
+2. :code:`/entrypoint.sh`: this entrypoint script will run :code:`jupyterhub-singleuser`. It is intended for use in the cloud JupyterHub deployment and should not be used locally.
+3. :code:`bash`: by specifying :code:`bash` (or any other shell installed in the container), you will get command-line access to the container, instead of a Jupyter environment.
+
+At this point, you should either have a Jupyter URL that you can use to access Jupyter, or you should have shell access to the container.
+You can now do whatever local testing you want of the image.
+
+------------------------------------
+Deploying the Tutorial to Kubernetes
+------------------------------------
+
+TBA

2025-HPDC/docker/Dockerfile.spawn

@@ -35,7 +35,9 @@ RUN chmod -R 777 ~/ ${HOME}
 WORKDIR ${HOME}
 
 COPY ./docker/spawn-entrypoint.sh /entrypoint.sh
+COPY ./docker/spawn-local-entrypoint.sh /local-entrypoint.sh
 RUN chmod 777 /entrypoint.sh
+RUN chmod 777 /local-entrypoint.sh
 
 USER ${NB_USER}
 

2025-HPDC/docker/spawn-local-entrypoint.sh

@@ -0,0 +1,3 @@
+#!/usr/bin/env bash
+
+/opt/global_py_venv/bin/jupyter-lab --ip=0.0.0.0