Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add documentation for running MESSAGEix on UniCC #279

Draft
wants to merge 9 commits into
base: main
Choose a base branch
from
Draft

Conversation

measrainsey
Copy link
Contributor

@measrainsey measrainsey commented Jan 16, 2025

WIP: migrate my internal guide and documentation on how to run MESSAGEix on the cluster (UniCC) and make some modifications

How to review

For @khaeru and/or @glatterf42 : Read the diff and note that the CI checks all pass.

PR checklist

  • Continuous integration checks all ✅
  • Add or expand tests; coverage checks both ✅
  • Add, expand, or update documentation.
  • Update doc/whatsnew.

Copy link

codecov bot commented Jan 16, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 76.7%. Comparing base (c782c30) to head (6b4cd85).

Additional details and impacted files
@@           Coverage Diff           @@
##            main    #279     +/-   ##
=======================================
- Coverage   77.6%   76.7%   -1.0%     
=======================================
  Files        211     211             
  Lines      16079   16079             
=======================================
- Hits       12481   12333    -148     
- Misses      3598    3746    +148     

see 7 files with indirect coverage changes

Copy link
Member

@glatterf42 glatterf42 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this extensive guide! Skimming over it, I thought of just a few notes :)

Comment on lines +499 to +503
GAMS_Demo,_for_EULA_and_demo_limitations_see___G240131/0001CB-GEN
https://www.gams.com/latest/docs/UG%5FLicense.html_______________
1496554900_______________________________________________________
0801346905_______________________________________________________
DC0000_______g_1_______________________________C_Eval____________
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even if it's a demo, I'm not sure we should post the contents to our docs.

#SBATCH --ntasks=1
#SBATCH --cpus-per-task=16

I usually request lots of run time (20 hours) and lots of memory (100 GB) because I don't want my job to for those reasons.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
I usually request lots of run time (20 hours) and lots of memory (100 GB) because I don't want my job to for those reasons.
I usually request lots of run time (20 hours) and lots of memory (100 GB) because I don't want my job to fail for those reasons.

sacct --format=jobid,MaxRSS,MaxVMSize,start,end,CPUTimeRAW,NodeList


Resources to request for reducing MESSAGEix run time
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's great to have a section like this, but if the cluster becomes more heavily used, there are more considerations for setting these parameters: in general, slurm will try to optimize its load so that it's running almost always at full capacity. This makes it harder to request lots of resources for extended amounts of time because the system will not be able to squeeze in such a task with other tasks, thus it will likely take longer for the task to be picked up.
I don't know if UniCC is already experiencing the kind of load to make these considerations necessary, but if/once it does, we would likely want to optimize these parameters further to request the minimum necessary that still reduces the run time :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants