refactor Dockerfile to use spack.yaml

[vsoch]

The build is locally working but not in the CI? My current setup is a bit of a hack, I'm going to try using spack as a more "streamlined" way of doing it (we hope!)

[jonesholger]

@vsoch Adrien is deep into updating RAJA's CI to use Spack v0.19.0 which had an API change for propagating some compiler flags. See
llnl/RAJAPerf#280.
llnl/RAJA#1391
llnl/RAJA#1384

Also v0.19.0 is minimum necessary to install caliper@master since it's Caliper package.py is updated to handle +rocm , and RAJAPerf's package.py

depends_on('caliper@master',when='+caliper')
depends_on('caliper@master +cuda',when='+caliper +cuda')
depends_on('caliper@master +rocm',when='+caliper +rocm')

CI will be using uberenv build, and we can try that here too
https://github.com/LLNL/RAJAPerf/tree/pr-from-fork/137#using-caliper

RAJAPerf's package.py is designed to output a hostconfig file which a subequent cmake -C can use to setup build files

The uberenv config has the phase stopping at hostconfig
{
"package_name" : "raja_perf",
"package_version" : "develop",
"package_final_phase" : "hostconfig",
"package_source_dir" : "../..",
"spack_url": "https://github.com/spack/spack.git",
"spack_commit": "bb8b4f9",
"spack_activate" : {},
"spack_configs_path": "tpl/RAJA/scripts/radiuss-spack-configs",
"spack_packages_path": "scripts/spack_packages",
"spack_concretizer": "clingo",
"spack_setup_clingo": false
}

https://github.com/LLNL/RAJAPerf/blob/pr-from-fork/137/scripts/spack_packages/raja_perf/package.py#L116

[vsoch]

okay thanks for the update! Not sure if I would have time to do work on this (have a lot of other things in the Q) but definitely open a PR if you have bandwidth!

[vsoch]

I've been working on making lab packages into Nix, and maybe could eventually try that for Raja too. <3

[jonesholger]

haha - now you have me looking at nix-shell ad-hoc Python environments; I'm constantly switching between 3.8 and 3.9. I should have time to work on this tutorial pretty soon. I've snagged myself into looking at Caliper's PAPI support, and some anomalous readings in a couple of places. I need to document this behavior. Hmm maybe we could play with PAPI here too.

[vsoch]

Definitely! So I can add those packages to my Nix TODO - I actually already have caliper :) So you are wanting in addition papi, and then maybe give umpire a shot? I'm enjoying making these so would be happy to, even it if doesn't hit this set of containers. I'm still learning good practices myself for Nix in containers (right now I have a base that is based on Ubuntu so I can still use apt-get in an emergency) https://github.com/rse-ops/nix-images/pkgs/container/nix-ubuntu and it's also designed so it works with vscode.

[jonesholger]

Yeah a Spack install of Caliper will bring in the PAPI lib among a bunch of other packages including MPI (likely openmpi in the container, and python too @3.10.8 unless we override - I'm concerned about Hatchet where I know 3.8 is a safe bet). Installation should be pain-free otherwise. For the demo it would be fun to test the gcc and clang OpenMP runtimes and plot results across OMP_NUM_THREADS=[2|4|8|16|32|etc], so we'll always build with openmp. We can do a ~mpi for the demo, which would minimize potential snags here with openssh and perl.

Maybe you could package up Hatchet - see how it goes.

RAJAPerf doesn't use Umpire - but I will mention that there is an effort to keep a Version set where RAJA, Umpire, and CHAI are known to work together. So yeah maybe we could remain cognizant about the bundling.

[vsoch]

Looks like we already have a version of papi in nix! https://github.com/NixOS/nixpkgs/blob/nixos-22.05/pkgs/development/libraries/science/benchmark/papi/default.nix#L23

I don't see hatchet, so I'll try that first.