Skip to content

Tool to launch serial processes on a cluster and aggregate results, using mpi4py

License

Notifications You must be signed in to change notification settings

benkirk/mpi4py_task_runner

Folders and files

NameName
Last commit message
Last commit date

Latest commit

679512d · May 1, 2024
Apr 22, 2024
Apr 19, 2024
Apr 27, 2024
May 1, 2024
Apr 26, 2024
Jun 10, 2019
Apr 13, 2019
May 11, 2021
Apr 20, 2019
May 14, 2019
Jun 20, 2019
Apr 18, 2019
Apr 26, 2024
Jun 14, 2019
Apr 16, 2019
Apr 24, 2019
Jun 20, 2019
Jun 15, 2019
Jun 20, 2019
May 9, 2019

Repository files navigation

mpi4py_task_runner

Tool to launch serial processes on a cluster and aggregate results, using mpi4py

Table of Contents

FSL europa

FSL

quickstart

# on a login node
mkdir /nobackup/$USER/codes
cd /nobackup/$USER/codes
git clone https://github.com/benkirk/mpi4py_task_runner.git
cd mpi4py_task_runner/
git checkout -b 0.1.3 v0.1.3
srun -n 80 --pty /bin/bash  # grab an interactive SLURM session on 80 cores

# then in the interactive shell
. utils/config_env_fsl.sh
mpiexec ./run.py
exit

# back on the login node
make list

# cat the file 'summary.txt'
make summary

# only if you really need to!!
make extract

# cleanup
git clean -xdf .

europa

Configure Environment

### Configure Environment
# Anaconda 3-5.0.1; python3
module purge
module load gcc slurm
module load anaconda/3-5.0.1
module load mpt
export PYTHONPATH=/software/x86_64/mpi4py/3.0.1-${MPI_ID_STRING}/lib/python3.6/site-packages

#####################
echo $PYTHONPATH
which mpiexec
which python

Execution

srun -n 64 --constraint=sky --pty /bin/bash -l # get an interactive shell, 2 nodes, 32 cores ea.
. utils/config_env_fsl.sh
mpiexec ./run.py

About

Tool to launch serial processes on a cluster and aggregate results, using mpi4py

Resources

License

Stars

Watchers

Forks

Packages

No packages published