Perl to Python port for general scripts

[abensonca]

It would be useful to be able to drop all Perl dependencies in Galacticus - both because Perl support is non-ideal (particularly on Mac OS), and because Python is much more widely used. Several scripts utilize Perl and should be ported to Python. The list below excludes those that are part of the build infrastructure - that will be treated as a separate project.

• ./doc/Bibliography_Demangle.pl
• ./scripts/aux/Split_Models_For_Workers.pl - obsolete - can be removed
• ./scripts/build/postprocessLinker.pl
• ./scripts/aux/workflowStatus.pl
• ./scripts/aux/backtraceSlurm.pl
• ./scripts/aux/archive.pl
• ./scripts/aux/bibliographyUpdate.pl
• ./scripts/aux/slocReport.pl
• ./scripts/build/libraryInterfaces.pl
• ./scripts/build/findExecutables.pl
• ./scripts/build/staticRelinker.pl
• ./plots/cooling_function.pl
• ./scripts/build/codeDirectivesParse.pl
• ./scripts/build/deepCopyActions.pl
• ./scripts/build/useDependencies.pl
• ./scripts/build/stateStorables.pl
• ./scripts/build/moduleDependencies.pl
• ./scripts/build/libraryDependencies.pl
• ./scripts/build/libraryInterfacesDependencies.pl
• ./scripts/build/postprocess.pl
• ./scripts/build/includeDependencies.pl
• ./scripts/build/buildProfiler.pl
• ./scripts/build/enumerateOpenMPCriticalSections.pl
• ./scripts/build/parameterDependencies.pl
• ./scripts/build/sourceDigests.pl
• ./scripts/doc/extractData.pl
• ./scripts/doc/constants.pl
• ./scripts/doc/Code_Analyzer.pl
• ./scripts/aux/parametersMigrate.pl
• ./scripts/aux/functionClassReferencesDebug.pl
• ./scripts/aux/embeddedAnalyzer.pl
• ./scripts/aux/Merge_Models.pl
• ./scripts/aux/localLaunchWrapper.pl
• ./scripts/aux/formatModuleUses.pl
• ./scripts/aux/parametersFormat.pl
• ./scripts/aux/validateParameters.pl
• ./scripts/aux/updateMonitor.pl
• ./scripts/aux/linkChecker.pl
• ./scripts/aux/pinocchioToIrate.pl
• ./scripts/aux/Bolshoi_Trees_Grab.pl
• ./scripts/aux/launch.pl
• ./scripts/aux/workaroundChecker.pl
• ./scripts/aux/openMPCriticalWaitProfile.pl
• ./scripts/aux/Millennium_Lightcone_Grab.pl
• ./scripts/aux/staticAnalyzer.pl
• ./scripts/aux/Millennium_Trees_Grab.pl
• ./scripts/aux/spellChecker.pl
• ./scripts/aux/WMAP9_Parameter_Covariance.pl
• ./scripts/analysis/metaAnalysis.pl
• ./scripts/analysis/debugAnalyzer.pl
• ./scripts/filters/vega_offset_effective_lambda.pl
• ./scripts/parameters/cosmologicalParametersMonteCarlo.pl
• ./scripts/ssps/Convert_Maraston_SSPs_to_Galacticus.pl
• ./scripts/ssps/Convert_BC2003_SSPs_to_Galacticus.pl
• ./constraints/dataAnalysis/projectedCorrelationFunction_SDSS_z0.07_Hearin/covarianceMatrix.pl
• ./constraints/dataAnalysis/stellarMassFunction_GAMA_z0.03/covarianceMatrix.pl
• ./constraints/dataAnalysis/stellarMassFunction_GAMA_z0.03/massDistanceRelation.pl
• ./constraints/dataAnalysis/stellarMassFunctions_ZFOURGE_z0.2_2.5/massRedshiftRelation.pl
• ./constraints/dataAnalysis/stellarMassFunctions_ZFOURGE_z0.2_2.5/covarianceMatrix.pl
• ./constraints/dataAnalysis/stellarMassFunctions_PRIMUS_z0_1/massRedshiftRelation.pl
• ./constraints/dataAnalysis/stellarMassFunctions_PRIMUS_z0_1/covarianceMatrix.pl
• ./constraints/dataAnalysis/stellarMassFunctions_PRIMUS_z0_1/massErrors.pl
• ./constraints/dataAnalysis/stellarMassFunctions_ULTRAVISTA_z0.2_4.0/massRedshiftRelation.pl
• ./constraints/dataAnalysis/stellarMassFunctions_ULTRAVISTA_z0.2_4.0/makeMask.pl
• ./constraints/dataAnalysis/stellarMassFunctions_ULTRAVISTA_z0.2_4.0/covarianceMatrix.pl
• ./constraints/dataAnalysis/stellarMassFunctions_UKIDSS_UDS_z3_5/completeness.pl
• ./constraints/dataAnalysis/stellarMassFunctions_UKIDSS_UDS_z3_5/covarianceMatrix.pl
• ./constraints/dataAnalysis/stellarMassFunctions_UKIDSS_UDS_z3_5/massLuminosityRelation.pl
• ./constraints/dataAnalysis/stellarMassFunctions_UKIDSS_UDS_z3_5/massErrors.pl
• ./constraints/dataAnalysis/scripts/covarianceMatrixProjectedCorrelation.pl
• ./constraints/dataAnalysis/scripts/generateCovarianceMatrixProjectedCorrelation.pl
• ./constraints/dataAnalysis/scripts/covarianceMatrix.pl
• ./constraints/dataAnalysis/scripts/generateCovarianceMatrix.pl
• ./constraints/dataAnalysis/stellarMassFunctions_VIPERS_z0_1/covarianceMatrix.pl
• ./constraints/dataAnalysis/stellarMassFunctions_VIPERS_z0_1/massDistanceRelation.pl
• ./constraints/dataAnalysis/stellarMassFunction_SDSS_z0.07_Bernardi/profileSystematic.pl
• ./constraints/dataAnalysis/stellarMassFunction_SDSS_z0.07_Bernardi/covarianceMatrix.pl
• ./constraints/dataAnalysis/stellarMassFunction_SDSS_z0.07_Bernardi/massDistanceRelation.pl
• ./constraints/dataAnalysis/stellarMassFunction_SDSS_z0.07/profileSystematic.pl
• ./constraints/dataAnalysis/stellarMassFunction_SDSS_z0.07/covarianceMatrix.pl
• ./constraints/dataAnalysis/stellarMassFunction_SDSS_z0.07/massLuminosityRelation.pl
• ./constraints/dataAnalysis/stellarMassFunction_SDSS_z0.07/satelliteHodSlopePrior.pl
• ./constraints/dataAnalysis/hiMassFunction_ALFALFA_z0.00/covarianceMatrix.pl
• ./constraints/dataAnalysis/hiMassFunction_ALFALFA_z0.00/lineWidthMassRelation.pl
• ./constraints/dataAnalysis/hiMassFunction_ALFALFA_z0.00/alfalfaHIMassErrorModel.pl
• ./constraints/posteriorPredictiveChecks.pl
• ./constraints/fitMultivariateNormal.pl
• ./constraints/scripts/wmap9CosmologyParametersConfig.pl
• ./constraints/scripts/planckCosmologyParametersConfig.pl
• ./constraints/convergenceGelmanRubin.pl
• ./constraints/acceptanceRate.pl
• ./constraints/projectionPursuit.pl
• ./constraints/maximumLikelihoodModel.pl
• ./constraints/visualization/mcmcVisualize.pl
• ./constraints/visualization/mcmcVisualizeTriangle.pl
• ./constraints/pipelines/darkMatter/haloMassFunctionPostProcess.pl
• ./constraints/pipelines/darkMatter/progenitorMassFunctionPostProcess.pl
• ./constraints/pipelines/darkMatter/pipeline.pl
• ./constraints/pipelines/darkMatter/spinConcentrationPreProcess.pl
• ./constraints/pipelines/darkMatter/progenitorMassFunctionPreProcess.pl
• ./constraints/pipelines/darkMatter/haloMassFunctionPreProcess.pl
• ./constraints/pipelines/darkMatter/haloMassFunctionGenerateContent.pl
• ./constraints/pipelines/darkMatter/haloMassFunctionZoomInExtract.pl
• ./constraints/pipelines/darkMatter/spinConcentrationPostProcess.pl
• ./testSuite/test-methods.pl
• ./testSuite/test-galacticStructureStateDeallocateBug.pl
• ./testSuite/test-regressions.pl
• ./testSuite/test-haloTriaxiality.pl
• ./testSuite/test-treeFilterLabels.pl
• ./testSuite/test-splitForests.pl
• ./testSuite/test-mass-definitions.pl
• ./testSuite/test-stateRestore.pl
• ./testSuite/test-bar-instability.pl
• ./testSuite/test-allowed-parameters.pl
• ./testSuite/test-branchSubsampling.pl
• ./testSuite/test-impulsiveHeating.pl
• ./testSuite/test-checkpointing.pl
• ./testSuite/test-concentration-Johnson2021.pl
• ./testSuite/test-mass-conservation-coldMode.pl
• ./testSuite/test-mass-host-maximum.pl
• ./testSuite/test-all.pl
• ./testSuite/test-outputs.pl
• ./testSuite/benchmark-all.pl
• ./testSuite/test-interoutput-star-formation-rate.pl
• ./testSuite/test-methods_MPI.pl
• ./testSuite/test-inactiveNumerics.pl
• ./testSuite/test-tidallyTruncatedNFWFit.pl
• ./testSuite/test-transferFunctionHalfModeSlope.pl
• ./testSuite/test-CGM-mass-cooled.pl
• ./testSuite/test-branchlessTrees.pl
• ./testSuite/test-merger-tree-write.pl
• ./testSuite/test-mass-conservation-standard.pl
• ./testSuite/test-tidalTracks.pl
• ./testSuite/test-Bolshoi-tree-builder.pl
• ./testSuite/test-constrained-merger-trees.pl
• ./testSuite/test-output.pl
• ./testSuite/scripts/decayingDarkMatterRetention.pl
• ./testSuite/test-mass-conservation-simple.pl
• ./testSuite/benchmarks.quickTest.pl
• ./testSuite/test-satellite-distance-minimum.pl
• ./testSuite/test-transfer-functions-axion.pl
• ./testSuite/test-output-times.pl
• ./testSuite/test-parameter-validation.pl
• ./testSuite/test-Millennium-tree-builder.pl
• ./testSuite/test-stateRestore_MPI.pl
• ./testSuite/test-memory-leaks.pl
• ./testSuite/test-radiativeTransfer-StromgrenSphere_MPI.pl
• ./testSuite/test-concentration-Ludlow2016.pl
• ./testSuite/benchmark-milkyWay.pl
• ./testSuite/test-extract-directives.pl
• ./testSuite/test-stellar-mass-weighted-ages.pl
• ./testSuite/test-model-integration.pl
• ./testSuite/test-reproducibility.pl
• ./testSuite/test-outputSelector.pl
• ./testSuite/benchmark-darkMatterOnlySubhalos.pl
• ./testSuite/regressions/setProperties.pl
• ./testSuite/regressions/mergerTreeBoxSizeWeight.pl
• ./testSuite/regressions/bug1066052.pl
• ./testSuite/regressions/cosmicEmu.pl
• ./testSuite/regressions/satellitePresetBoundMassNonZero.pl
• ./testSuite/test-noninstantaneous-recycling.pl
• ./testSuite/test-parameter-migration.pl
• ./testSuite/test-cosmology.pl
• ./testSuite/test-inactive_luminosities.pl
• ./testSuite/test-parallelTreeBuild.pl
• ./testSuite/test-constraint-mass-function.pl
• ./testSuite/test-fortran-utils.pl
• ./testSuite/test-constraint-deterministic-spins.pl
• ./testSuite/test-merger-tree-builder.pl
• ./testSuite/test-duplicatedOutputPropertyName.pl
• ./testSuite/test-output-tree-contiguousity.pl
• ./testSuite/test-perl-modules.pl

[abensonca]

Claude's opinion:

Here are the best places to start, ranked by simplicity:

Top Recommendations for Perl-to-Python Porting

Easiest (No dependencies, pure logic)

1. scripts/build/postprocessLinker.pl (~32 lines)

   • Filters linker warnings from stdin using regex
   • Zero external dependencies — pure text processing
   • Maps cleanly to Python's sys.stdin, re, and sys.exit

2. scripts/aux/backtraceSlurm.pl (~50 lines)

   • Extracts stack traces from Slurm job threads using scontrol, srun, gdb
   • Only uses system commands, no Perl library deps
   • Maps directly to Python's subprocess

Easy (Standard library dependencies only)

3. scripts/aux/Split_Models_For_Workers.pl (~37 lines)

   • Splits a Galacticus XML parameter file across multiple workers
   • Uses XML::Simple → Python's xml.etree.ElementTree or lxml

4. scripts/aux/localLaunchWrapper.pl (~34 lines)

   • Launches scripts in parallel using Perl threads
   • Maps to Python's threading or concurrent.futures

5. scripts/aux/slocReport.pl (~97 lines)

   • Counts source lines of code across languages, handles embedded XML/LaTeX
   • Uses File::Find + JSON → Python's os.walk + json

Medium (Some internal module deps)

6. scripts/aux/validateParameters.pl (~111 lines)

   • Validates XML parameter files against a schema
   • Uses XML validation libs → Python's lxml with schema validation

7. scripts/aux/openMPCriticalWaitProfile.pl (~37 lines)

   • Reads HDF5 metadata and reports OpenMP timing stats
   • Uses PDL (Perl Data Language) → Python's h5py + numpy

---

My suggestion: Start with postprocessLinker.pl — it's self-contained, has no dependencies, and the porting will be straightforward. Then move to Split_Models_For_Workers.pl since XML manipulation is a common pattern you'll encounter elsewhere in the repo.