Skip to content

Standalone reweighting: optimise cloning of shared lib #24

@ajgilbert

Description

@ajgilbert

Currently standalone_reweight.pywill make N copies of the allmatrix2py.so shared lib, where N = number of reweighting points. This is a bit haphazard, and hardcodes these copies in the reweighting module directory. Issues:

  • parallel jobs might try and do these copy operations in parallel
  • re-use of existing copies seems to have been disabled
  • better approach: make cache directory user-customisable, perhaps defaulting to a tmp location
  • consider adding support for sandboxfs, to avoid making actual copies. We would start a sandboxfs mount in $TMPDIR, configured with as many copies of the shared lib as needed, all pointing to the original file. E.g. this works (./bin/sandboxfs --mapping ro:/testdir/rw0/allmatrix2py.so:/afs/cern.ch/work/a/agilbert/projects/leshouches/EFT2Obs/rw_ggF-SMEFTatNLO-1jet-loop/rwgt/rw_me_second/SubProcesses/allmatrix2py.so --mapping ro:/testdir/rw1/allmatrix2py.so:/afs/cern.ch/work/a/agilbert/projects/leshouches/EFT2Obs/rw_ggF-SMEFTatNLO-1jet-loop/rwgt/rw_me_second/SubProcesses/allmatrix2py.so $TMPDIR/testmount), but only on lxplus8.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions