CME — Caterpillar Made Easy

An interactive desktop GUI for setting up, running and inspecting cosmological zoom-in simulations.

CME wraps the entire zoom-in resimulation pipeline — candidate halo selection, Lagrangian region extraction, MUSIC initial conditions, P-Gadget3 parameter files, cluster job submission, and contamination analysis — behind a single Traits/TraitsUI window, so a whole suite of resimulations can be configured and launched without writing code. It was built for the Caterpillar Project at MIT/Harvard, but the workflow applies to any suite built on MUSIC + Gadget + Rockstar.

Important

The GUI cannot run from a clean checkout. Every tab imports a modules package of simulation I/O readers that is not distributed with this repository — see Prerequisites. Launching the interface also needs a display, a built MUSIC and P-Gadget3, and Rockstar catalogues for a parent simulation.

What does work standalone is everything under Library: the path conventions, the cosmologies, the candidate selection, and the MUSIC/Gadget/scheduler file writers are display-free, pip-installable and covered by 171 tests. The code was Python 2 until recently; see AUDIT.md for the assessment that drove the modernisation and what remains outstanding.

---

Features

Stage	Tab	What it does
Dependencies	Install	Builds FFTW2/3, HDF5, GSL, zlib and szip into a local lib/ prefix.
Candidate selection	Candidates	Scans a parent Rockstar catalogue for Milky-Way analogues using a mass range plus three tunable isolation ("exclusion zone") criteria; tabulates and exports the surviving sample.
Lagrangian regions	Initial Conditions	Traces a candidate's z=0 particles back to the parent ICs, measures the Lagrangian centre/extent/volume, visualises the region in any projection, and writes the MUSIC region point file.
Initial conditions	Initial Conditions	Generates MUSIC .conf files across a grid of resolutions, paddings, overlaps, region shapes and cosmologies, then invokes MUSIC for each combination.
Simulation setup	Gadget	Exposes every P-Gadget3 param.txt and Config.sh option, auto-scales gravitational softenings with levelmax, and designs the snapshot output list in expansion factor.
Progress tracking	Gadget → Existence	Tabulates which ICs, snapshots and halo catalogues already exist across the whole parameter sweep.
Job submission	Gadget → Write & Submit	Writes and submits PBS/Torque or SLURM run scripts, optionally recompiling Gadget first.
Halo analysis	Halo Finder	Plots any pair of Rockstar quantities for a parent box or a zoom, restricted to hosts, subhaloes or all; renders 3D halo distributions and velocity fields in Mayavi.
Merger histories	Merger Tree	Plots main-branch evolution of any tracked quantity from consistent-trees output.
Resolution quality	Contam.	Measures low-resolution particle contamination versus radius from the target FOF group, and maps it as a 2D heatmap in any projection or a 3D Mayavi scene.

Cosmologies available throughout: WMAP1, WMAP3, WMAP5, WMAP7, WMAP9, PLANCK.

---

Screenshots

Candidate selection	Lagrangian region inspection
Constructing ICs across a sweep	Merger tree inspection
Halo distributions in 3D (Mayavi)	Halo velocity field in 3D (Mayavi)
Inter-halo relations from Rockstar	Contamination versus radius
Contamination heatmap	P-Gadget3 parameters
Snapshot output list	Existence check across the sweep
Submit to PBS or SLURM

---

Prerequisites

Python environment

Python 3.9 or newer. Dependencies are declared in pyproject.toml and split by what you need:

Extra	Contents	Needed for
(base)	numpy, pandas	the display-free library and the test suite
gui	traits, traitsui, enable, mayavi, matplotlib, wxPython	launching the interface
dev	pytest, ruff	running tests and linting

The GUI uses the modern traits / traitsui packages with the wxPython toolkit. A Qt port is the obvious next step but has not been done — see AUDIT.md.

External I/O package (not included)

Every module imports a modules package that must be on PYTHONPATH. Without it, nothing runs. It provides:

Import	Provides
modules.readsnapshots.readsnap	Gadget binary snapshot reader (read_block)
modules.readsnapshots.readsnapHDF5	Gadget HDF5 reader (read_block, snapshot_header)
modules.readsnapshots.readids	Particle ID reader
modules.readhalos.RSDataReaderv2	Rockstar catalogue reader (RSDataReader)
modules.readhalos.readsubf	SUBFIND group catalogue reader (subfind_catalog)
modules.mergertrees.MTCatalogue	consistent-trees merger tree reader
modules.convertfiles.reWriteIC	getBlocks() — post-processes MUSIC output for Gadget
modules.brendanlib.grifflib	Helpers: getcandidatelist, getcentext, getlagrxyz, drawcircle, addsubtitle, placenormtext, CorrectPos, COM. (cosmoconstant is no longer needed — see cme.cosmology.)

External codes and data

• MUSIC (Hahn & Abel 2011) — compiled binary, path set on the Home tab
• P-Gadget3 — source tree at <home>/lib/P-Gadget3, compiled from the Gadget tab
• Rockstar halo catalogues and consistent-trees merger trees for the parent box
• A parent simulation (the Caterpillar 512³, 100 Mpc/h box) with both outputs/ snapshots and ics/
• A PBS/Torque or SLURM cluster, for job submission

---

Installation

git clone https://github.com/bgriffen/cme.git
cd cme

# The display-free library and the test suite:
pip install -e ".[dev]"
python -m pytest tests/

# To also launch the interface:
pip install -e ".[gui]"
# The simulation I/O readers are a separate, undistributed dependency:
export PYTHONPATH=/path/to/modules-parent:$PYTHONPATH

Usage

cme          # or: python -m cme.app

The window splits into a plot panel (matplotlib, plus a Mayavi scene on its own tab) on the left and the workflow tabs on the right. Marker colour, style and size at the bottom left apply to every 2D plot. Work left to right through the tabs:

1. Home — pick a cluster profile and confirm the derived paths to MUSIC, P-Gadget3, the parent simulation and the project data directory. Everything downstream derives from these. Profiles come from clusters.toml and are matched against the hostname automatically.
2. Candidates — either load an existing candidates.dat or run the selection, then add the IDs you want into the working halo sample. That sample propagates to every other tab.
3. Initial Conditions — generate and inspect the Lagrangian region per halo and nrvir value, then produce the MUSIC configs and run them.
4. Gadget — set parameters, build the output list, then write and submit the run scripts.
5. Halo Finder / Merger Tree / Contam. — analyse the results.

Run directory naming

Every resimulation lives at <project data>/halos/H<haloid>/<foldername>/, where the folder name encodes the full parameter set:

H<haloid>_B<B|E>_Z<zinit>_P<padding>_LN<levelmin>_LX<levelmax>_O<overlap>_NV<nrvir>

For example, H190897_BE_Z127_P7_LN7_LX11_O4_NV4 is halo 190897, ellipsoidal region, starting at z=127, padding 7, levels 7→11, overlap 4, region enclosing 4·R_vir(z=0).

Lagrangian region files are written to <project data>/ics/lagr/H<haloid>NRVIR<n>, each alongside a .head file holding the region centre and extent.

Project structure

cme/
├── clusters.toml            # Cluster profiles: hostnames, paths, scheduler settings
├── pyproject.toml
├── src/cme/
│   ├── app.py               # Entry point; assembles the tabbed window
│   ├── naming.py            # Run-directory and Lagrangian path conventions
│   ├── cosmology.py         # Named cosmologies (WMAP1-9, PLANCK)
│   ├── config.py            # Loads clusters.toml
│   ├── shell.py             # Runs external commands without a shell
│   ├── science/
│   │   └── candidates.py    # Isolation-criteria halo selection
│   ├── writers/
│   │   ├── music.py         # MUSIC .conf generation
│   │   ├── gadget.py        # param.txt, Config.sh, ExpansionList, softenings
│   │   └── submit.py        # PBS and SLURM submission scripts
│   └── ui/                  # One HasTraits tab per module
│       ├── common.py        #   Shared imports, matplotlib-in-TraitsUI editor
│       ├── home.py  install.py  candidates.py  ics.py
│       └── gadget.py  halos.py  mergertree.py  contamination.py
├── tests/
└── screenshots/

Each tab is a HasTraits subclass instantiated by ApplicationMain and handed a reference to it, so tabs share state (the halo sample, the base paths, the active figure) through the self.main back-reference.

Library

Everything outside cme/ui/ is display-free: plain functions over arrays and scalars, with no Traits, matplotlib or filesystem dependency. That makes the pipeline scriptable for a large suite and testable without a cluster.

from cme import naming, cosmology
from cme.writers import music, gadget, submit
from cme.science.candidates import select, ExclusionZone

# The run-directory convention, in one place
naming.run_dir_name(halo_id=190897, region="ellipsoid", padding=7,
                    levelmin=7, levelmax=11, overlap=4, nrvir=4)
# 'H190897_BE_Z127_P7_LN7_LX11_O4_NV4'

# Isolation criteria: "no halo above mass M within distance R"
select(hosts, mass_range=(7e11, 7e12), hubble=0.6711, zones=[
    ExclusionZone(distance=1.4, mass_multiple=0.5),   # half my mass, 1.4 Mpc
    ExclusionZone(distance=3.0, mass=7e12),
    ExclusionZone(distance=4.0, mass=7e13),
])

# Configuration files as strings, so they can be diffed and asserted on
music.resim_conf(cosmo=cosmology.PLANCK, boxlength=100, levelmax=11, ...)
gadget.param_file(cosmo=cosmology.PLANCK, boxsize=100.0, levelmax=11)
submit.slurm_script(job_name="H19EP7L11N4", cores=512, ...)

Cluster profiles

clusters.toml describes each machine: which hostnames select it, where home directories live, and where MUSIC, P-Gadget3 and the project data sit, using {home}, {user} and {master} placeholders.

[clusters.odyssey]
hostnames = ["rclogin13.rc.fas.harvard.edu"]
home_root = "/n/home01"
gadget = "{master}/lib/P-Gadget3"
data = "{home}/caterpillar"
scheduler = "SLURM"
queue = "itc_cluster"

Resolution order: $CME_CLUSTERS, then ~/.config/cme/clusters.toml, then the copy in the repository. A profile whose hostnames list is empty must be selected by hand on the Home tab. With no file found at all, a single local profile under your home directory is used, so the application still starts.

Testing

pip install -e ".[dev]"
python -m pytest tests/          # 171 tests
python -m ruff check .

The suite covers the display-free library only — the naming conventions, the cosmologies, the candidate selection, the config-file writers and the cluster-profile loader — because those are the parts that can run without a display, a cluster or the undistributed modules package. Several tests are pinned regressions for specific defects; they cite the relevant AUDIT.md section in a comment.

CI additionally runs the doctests, ruff format --check over the library, and compileall over the GUI modules on Python 3.9 and 3.12 — those cannot be imported in CI, but they must at least parse.

Known limitations

• The GUI cannot be launched from a clean checkout, because the modules I/O package is not distributed here. This is the single biggest barrier to reuse.
• The GUI is untested. 171 tests cover the library; the tabs themselves have no coverage, since exercising them needs a display and a real parent simulation. Changes to cme/ui/ are verified only by inspection and by compileall.
• wxPython is legacy. TraitsUI still supports it, but Qt (PySide6) is the practical default and a port is the obvious next step.
• The Install tab builds FFTW2, HDF5 1.8 and GSL 1.9 from a lib/installs/ source tree that is also not in the repository. Use your system package manager instead.
• Two parent-simulation paths remain hardcoded in cme/ui/ics.py (PARENT_SNAPSHOT, PARENT_ICS); they belong in clusters.toml.
• The tabs still share state through a self.main back-reference, which makes their construction order load-bearing.

Acknowledgments

Initial conditions are generated with MUSIC — Hahn & Abel (2011), Multi-scale initial conditions for cosmological simulations, MNRAS 415, 2101. Simulations are run with P-Gadget3, descended from GADGET-2 — Springel (2005), MNRAS 364, 1105. Halo finding uses Rockstar — Behroozi, Wechsler & Wu (2013), ApJ 762, 109 — and merger trees use consistent-trees — Behroozi et al. (2013), ApJ 763, 18.

Developed as part of the Caterpillar Project at MIT/Harvard.

Contact

Brendan Griffen — contact+cme@grifflabs.dev

License

MIT — see LICENSE.