Add .util.slurm and CLI

[khaeru]

Building on #279, some simple utility code could reduce the difficulty of using message-ix-models with SLURM on UniCC and elsewhere. I imagine the following as minimum functionality:

• The user runs a command like mix-models sbatch [options] -- transport run --nodes=R12 --from="SSP2 built" "SSP2 solved" in which:

  • mix-models sbatch is the entry point.
  • [options] control the interaction with SLURM/sbatch.
  • transport run … are the commands to be run.

• The code generates (maybe using a Jinja2 template) a job file in the user's cache/temporary working directory that resembles:

  #!/bin/sh
  #SBATCH --time=3:00:00
  #SBATCH --mem=40G
  #SBATCH --mail-type=BEGIN,END,FAIL
  #SBATCH [email protected]
  #SBATCH -o ~/out/solve_%J.out
  #SBATCH -e ~/err/solve_%J.err
  
  module purge
  source /opt/apps/lmod/8.7/init/bash
  module load Python/3.11.5-GCCcore-13.2.0
  module load Java
  
  uv venv …    
    
  mix-models transport run --nodes=R12 --from="SSP2 built" "SSP2 solved
  

  …in which:

  • The #SBATCH lines are assembled from configuration; the CLI [options]; and common defaults.
  • The lines like module purge and other preamble are from configuration.
  • The uv venv line creates a node-specific virtual environment and installs message-ix-models and its dependencies. This could also work by activating a particular virtual environment know to already exist on the node; this would reduce start-up time but also risk of file collisions etc.
  • The line mix-models transport run … exactly matches the invocation on the command line.

• The code invokes sbatch job.sh and returns any output to the user.

This could later be extended to, for instance, run multiple tasks like #SBATCH --ntasks=10 / srun --ntasks=1 mix-models transport …. This would require generating a list of individual tasks.