From 42bdf2ebaeda160e9d7cda433ca2c584bf2b2852 Mon Sep 17 00:00:00 2001 From: Simon Gravelle Date: Tue, 24 Sep 2024 10:11:28 +0200 Subject: [PATCH 1/8] started collecting multiple job handeling --- docs/source/oar.rst | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/source/oar.rst b/docs/source/oar.rst index 2a94c9b..911e81c 100644 --- a/docs/source/oar.rst +++ b/docs/source/oar.rst @@ -40,3 +40,6 @@ You can cancel all your jobs at once by typing: where username should be replaced by your ID. +Launch Multiple Jobs +-------------------- + From ab8c122fb9f650de20c08d39f0ef47d1fcdf3a8d Mon Sep 17 00:00:00 2001 From: Simon Gravelle Date: Tue, 24 Sep 2024 10:29:21 +0200 Subject: [PATCH 2/8] upadated oar --- docs/source/oar.rst | 56 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 56 insertions(+) diff --git a/docs/source/oar.rst b/docs/source/oar.rst index 911e81c..1c45ff6 100644 --- a/docs/source/oar.rst +++ b/docs/source/oar.rst @@ -43,3 +43,59 @@ where username should be replaced by your ID. Launch Multiple Jobs -------------------- +Here, tips to efficiently launch multiple jobs are provided. + +Passing arguments +_________________ + +Assuming that one has the job submission script named *sub.sh* with name *lmp-myvariable-0*, +that launch a lammps input script named *input.lmp* by passing a variable +named *myvariable* into it: + +.. code:: bash + + #!/bin/bash + #OAR -n lmp-myvariable-0 + #OAR -l /nodes=1/cpu=1/core=4,walltime=48:00:00 + #OAR --stdout log-water.out + #OAR --stderr log-water.err + #OAR --project tamtam + + # Path to the LAMMPS MPI executable + lmp=/path/lmp_mpi + + myvariable=0 + + # Run LAMMPS using MPI, with 4 processes, using the input from 'input.lmp' + mpirun -np 4 ${lmp} -in input.lmp -var nb2 ${myvariable} + +If one wants to launch the current job, one simply have to type: + +.. code:: bash + + chmod +x sub.sh + oarsub -S ./sub.sh + +and a single job with name *lmp-myvariable-0* will be send. +To launch multiple simulations with different values of *myvariable*, one +can create a second bash script containing: + +.. code:: bash + + #!/bin/bash + set -e # Exit immediately if error + + # Loop over a desired set of values for myvariable + for myvariable in 0 1 2 + do + # Create a new line that will replace the line containing `myvariable` in the script sub.sh + newline='myvariable='$myvariable + # Find the current line in sub.sh that contains 'myvariable =', storing it in the variable oldline + # This assumes there is exactly one such line, otherwise the behavior may be unexpected + oldline=$(cat sub.sh | grep 'myvariable =') + # Use sed to replace the old line with the new line (newline) in sub.sh + # This command searches for the line containing the value in oldline and replaces it with newline + sed -i '/'"$oldline"'/c\'"$newline" sub.sh + chmod +x sub.sh + oarsub -S ./sub.sh + done From 8262afcb58a9fb191889ef6cff0fe9ebcef023d9 Mon Sep 17 00:00:00 2001 From: Simon Gravelle Date: Tue, 24 Sep 2024 10:30:52 +0200 Subject: [PATCH 3/8] updated bash method --- docs/source/oar.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/oar.rst b/docs/source/oar.rst index 1c45ff6..b46b16b 100644 --- a/docs/source/oar.rst +++ b/docs/source/oar.rst @@ -45,8 +45,8 @@ Launch Multiple Jobs Here, tips to efficiently launch multiple jobs are provided. -Passing arguments -_________________ +Batch Job Submission +____________________ Assuming that one has the job submission script named *sub.sh* with name *lmp-myvariable-0*, that launch a lammps input script named *input.lmp* by passing a variable From 1d10896c25309fbf19095dc705b0073ccde28325 Mon Sep 17 00:00:00 2001 From: Simon Gravelle Date: Tue, 24 Sep 2024 10:36:43 +0200 Subject: [PATCH 4/8] updated multiple script --- docs/source/oar.rst | 31 ++++++++++++++++++++++--------- 1 file changed, 22 insertions(+), 9 deletions(-) diff --git a/docs/source/oar.rst b/docs/source/oar.rst index b46b16b..4d4f4a8 100644 --- a/docs/source/oar.rst +++ b/docs/source/oar.rst @@ -77,25 +77,38 @@ If one wants to launch the current job, one simply have to type: oarsub -S ./sub.sh and a single job with name *lmp-myvariable-0* will be send. -To launch multiple simulations with different values of *myvariable*, one -can create a second bash script containing: +To launch multiple simulations with different values of *myvariable*, +say 0, 1, and 2, one can create a second bash script, named *multi-sub.sh*, +and containing: .. code:: bash #!/bin/bash - set -e # Exit immediately if error + set -e - # Loop over a desired set of values for myvariable for myvariable in 0 1 2 do - # Create a new line that will replace the line containing `myvariable` in the script sub.sh + # deal with OAR -n + newline='#OAR -n lmp-myvariable-'$myvariable + oldline=$(cat sub.sh | grep '#OAR -n lmp-myvariable-') + sed -i '/'"$oldline"'/c\'"$newline" sub.sh + # deal with myvariable newline='myvariable='$myvariable - # Find the current line in sub.sh that contains 'myvariable =', storing it in the variable oldline - # This assumes there is exactly one such line, otherwise the behavior may be unexpected oldline=$(cat sub.sh | grep 'myvariable =') - # Use sed to replace the old line with the new line (newline) in sub.sh - # This command searches for the line containing the value in oldline and replaces it with newline sed -i '/'"$oldline"'/c\'"$newline" sub.sh chmod +x sub.sh oarsub -S ./sub.sh done + +The *newline* command creates a new line that will replace the line +containing *myvariable* in the script sub.sh +The *oldline=* command finds the current line in sub.sh that contains 'myvariable =', +storing it in the variable oldline. This assumes there is exactly one such line, +otherwise the behavior may be unexpected. Then, sed is used to replace the old +line with the new line (newline) in *sub.sh*. + +Then, simply run *multi-sub.sh* by typing: + +.. code:: bash + + bash multi-sub.sh \ No newline at end of file From e878fbfee59a7643c6686884c13f21ff6c61e274 Mon Sep 17 00:00:00 2001 From: Cecilia Herrero <119974731+ceciherr@users.noreply.github.com> Date: Tue, 24 Sep 2024 11:09:49 +0200 Subject: [PATCH 5/8] Update oar.rst --- docs/source/oar.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/source/oar.rst b/docs/source/oar.rst index 4d4f4a8..ec3e32c 100644 --- a/docs/source/oar.rst +++ b/docs/source/oar.rst @@ -50,7 +50,7 @@ ____________________ Assuming that one has the job submission script named *sub.sh* with name *lmp-myvariable-0*, that launch a lammps input script named *input.lmp* by passing a variable -named *myvariable* into it: +named *myvariable* into it. Additionally, here the job ID is used as an input for the random seed, allowing for example different initial configurations: .. code:: bash @@ -67,7 +67,7 @@ named *myvariable* into it: myvariable=0 # Run LAMMPS using MPI, with 4 processes, using the input from 'input.lmp' - mpirun -np 4 ${lmp} -in input.lmp -var nb2 ${myvariable} + mpirun -np 4 ${lmp} -in input.lmp -var nb2 ${myvariable} -var seedin $OAR_JOBID If one wants to launch the current job, one simply have to type: @@ -111,4 +111,4 @@ Then, simply run *multi-sub.sh* by typing: .. code:: bash - bash multi-sub.sh \ No newline at end of file + bash multi-sub.sh From 34b37de0ab022baddfcea3a79f14a52cf77422e5 Mon Sep 17 00:00:00 2001 From: Simon Gravelle Date: Tue, 24 Sep 2024 13:20:47 +0200 Subject: [PATCH 6/8] moved bash tip to lammps tips --- docs/source/lammps.rst | 68 ++++++++++++++++++++++++++++++++++++++++++ docs/source/oar.rst | 68 ++++-------------------------------------- 2 files changed, 74 insertions(+), 62 deletions(-) diff --git a/docs/source/lammps.rst b/docs/source/lammps.rst index 6cb82ec..70a309c 100644 --- a/docs/source/lammps.rst +++ b/docs/source/lammps.rst @@ -75,3 +75,71 @@ chmod and launch it using: chmod +x ./myfile.sh oarsub -S ./myfile.sh + +Launch multiple jobs using bash +_______________________________ + +Assuming that one has the job submission script named *sub.sh* with name *lmp-myvariable-0*, +that launch a lammps input script named *input.lmp* by passing a variable +named *myvariable* into it. Additionally, here the job ID is used as an input for the random seed, allowing for example different initial configurations: + +.. code:: bash + + #!/bin/bash + #OAR -n lmp-myvariable-0 + #OAR -l /nodes=1/cpu=1/core=4,walltime=48:00:00 + #OAR --stdout log-water.out + #OAR --stderr log-water.err + #OAR --project tamtam + + # Path to the LAMMPS MPI executable + lmp=/path/lmp_mpi + + myvariable=0 + + # Run LAMMPS using MPI, with 4 processes, using the input from 'input.lmp' + mpirun -np 4 ${lmp} -in input.lmp -var nb2 ${myvariable} -var seedin $OAR_JOBID + +If one wants to launch the current job, one simply have to type: + +.. code:: bash + + chmod +x sub.sh + oarsub -S ./sub.sh + +and a single job with name *lmp-myvariable-0* will be send. +To launch multiple simulations with different values of *myvariable*, +say 0, 1, and 2, one can create a second bash script, named *multi-sub.sh*, +and containing: + +.. code:: bash + + #!/bin/bash + set -e + + for myvariable in 0 1 2 + do + # deal with OAR -n + newline='#OAR -n lmp-myvariable-'$myvariable + oldline=$(cat sub.sh | grep '#OAR -n lmp-myvariable-') + sed -i '/'"$oldline"'/c\'"$newline" sub.sh + # deal with myvariable + newline='myvariable='$myvariable + oldline=$(cat sub.sh | grep 'myvariable =') + sed -i '/'"$oldline"'/c\'"$newline" sub.sh + chmod +x sub.sh + oarsub -S ./sub.sh + done + +The *newline* command creates a new line that will replace the line +containing *myvariable* in the script sub.sh +The *oldline=* command finds the current line in sub.sh that contains 'myvariable =', +storing it in the variable oldline. This assumes there is exactly one such line, +otherwise the behavior may be unexpected. Then, sed is used to replace the old +line with the new line (newline) in *sub.sh*. + +Then, simply run *multi-sub.sh* by typing: + +.. code:: bash + + bash multi-sub.sh diff --git a/docs/source/oar.rst b/docs/source/oar.rst index ec3e32c..1827df8 100644 --- a/docs/source/oar.rst +++ b/docs/source/oar.rst @@ -45,70 +45,14 @@ Launch Multiple Jobs Here, tips to efficiently launch multiple jobs are provided. -Batch Job Submission -____________________ - -Assuming that one has the job submission script named *sub.sh* with name *lmp-myvariable-0*, -that launch a lammps input script named *input.lmp* by passing a variable -named *myvariable* into it. Additionally, here the job ID is used as an input for the random seed, allowing for example different initial configurations: +.. SG: this will be completed when Romain send me an example. .. code:: bash #!/bin/bash - #OAR -n lmp-myvariable-0 - #OAR -l /nodes=1/cpu=1/core=4,walltime=48:00:00 - #OAR --stdout log-water.out - #OAR --stderr log-water.err - #OAR --project tamtam - - # Path to the LAMMPS MPI executable - lmp=/path/lmp_mpi - - myvariable=0 - - # Run LAMMPS using MPI, with 4 processes, using the input from 'input.lmp' - mpirun -np 4 ${lmp} -in input.lmp -var nb2 ${myvariable} -var seedin $OAR_JOBID - -If one wants to launch the current job, one simply have to type: - -.. code:: bash - - chmod +x sub.sh - oarsub -S ./sub.sh - -and a single job with name *lmp-myvariable-0* will be send. -To launch multiple simulations with different values of *myvariable*, -say 0, 1, and 2, one can create a second bash script, named *multi-sub.sh*, -and containing: - -.. code:: bash - - #!/bin/bash - set -e - - for myvariable in 0 1 2 - do - # deal with OAR -n - newline='#OAR -n lmp-myvariable-'$myvariable - oldline=$(cat sub.sh | grep '#OAR -n lmp-myvariable-') - sed -i '/'"$oldline"'/c\'"$newline" sub.sh - # deal with myvariable - newline='myvariable='$myvariable - oldline=$(cat sub.sh | grep 'myvariable =') - sed -i '/'"$oldline"'/c\'"$newline" sub.sh - chmod +x sub.sh - oarsub -S ./sub.sh - done - -The *newline* command creates a new line that will replace the line -containing *myvariable* in the script sub.sh -The *oldline=* command finds the current line in sub.sh that contains 'myvariable =', -storing it in the variable oldline. This assumes there is exactly one such line, -otherwise the behavior may be unexpected. Then, sed is used to replace the old -line with the new line (newline) in *sub.sh*. - -Then, simply run *multi-sub.sh* by typing: - -.. code:: bash + ... + #OAR --array 50 - bash multi-sub.sh + id=`echo "$OAR_JOB_ID - $OAR_ARRAY_ID + 2" | bc -l` # <---- This id runs from 0 to 50, each job gets one, with which you can e.g. index a bash array + temperatures=(0.25 0.5 1 .... ) + T=${temperatures[${id}]} \ No newline at end of file From 6e6c7dec450d4cb3fb1dfb038dbda27c9cb3f119 Mon Sep 17 00:00:00 2001 From: Simon Gravelle Date: Tue, 24 Sep 2024 13:22:58 +0200 Subject: [PATCH 7/8] small fix --- docs/source/lammps.rst | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/docs/source/lammps.rst b/docs/source/lammps.rst index 70a309c..011f9e7 100644 --- a/docs/source/lammps.rst +++ b/docs/source/lammps.rst @@ -64,7 +64,10 @@ Create a bash (``.sh``) file with the following content: #OAR --stderr log.err #OAR --project tamtam - mpirun -np 4 /PATH-TO-LAMMPS/lmp_mpi -in input.lmp + # Path to the LAMMPS executable + lmp=/path/lmp_mpi + + mpirun -np 4 ${lmp} -in input.lmp where ``input.lmp`` is your LAMMPS input file, and there the project was assumed to be `tamtam` (to adapt to your case). Here, 4 CPU cores are requested, @@ -81,18 +84,19 @@ _______________________________ Assuming that one has the job submission script named *sub.sh* with name *lmp-myvariable-0*, that launch a lammps input script named *input.lmp* by passing a variable -named *myvariable* into it. Additionally, here the job ID is used as an input for the random seed, allowing for example different initial configurations: +named *myvariable* into it. Additionally, here the job ID is used as an +input for the random seed, allowing for example different initial configurations: .. code:: bash #!/bin/bash #OAR -n lmp-myvariable-0 - #OAR -l /nodes=1/cpu=1/core=4,walltime=48:00:00 - #OAR --stdout log-water.out - #OAR --stderr log-water.err + #OAR -l /nodes=1/cpu=1/core=4,walltime=12:00:00 + #OAR --stdout log.out + #OAR --stderr log.err #OAR --project tamtam - # Path to the LAMMPS MPI executable + # Path to the LAMMPS executable lmp=/path/lmp_mpi myvariable=0 From f03f4b79ef9e23112b9e3f3b6419d2a3298be63a Mon Sep 17 00:00:00 2001 From: Simon Gravelle Date: Tue, 24 Sep 2024 13:24:08 +0200 Subject: [PATCH 8/8] small typo --- docs/source/lammps.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/source/lammps.rst b/docs/source/lammps.rst index 011f9e7..e2808c9 100644 --- a/docs/source/lammps.rst +++ b/docs/source/lammps.rst @@ -53,7 +53,7 @@ Then, from the ``src/`` folder, type: Run LAMMPS ---------- -Create a bash (``.sh``) file with the following content: +Create a bash file named *sub.sh* with the following content: .. code-block:: bash @@ -69,15 +69,15 @@ Create a bash (``.sh``) file with the following content: mpirun -np 4 ${lmp} -in input.lmp -where ``input.lmp`` is your LAMMPS input file, and there the project was assumed +where ``input.lmp`` is your LAMMPS input file, and where the project was assumed to be `tamtam` (to adapt to your case). Here, 4 CPU cores are requested, as well as a total duration of 12 hours. Then, make the file file executable with chmod and launch it using: .. code-block:: bash - chmod +x ./myfile.sh - oarsub -S ./myfile.sh + chmod +x ./sub.sh + oarsub -S ./sub.sh Launch multiple jobs using bash _______________________________