Expand & automate testing system #31
Description
As we discussed for our RESPEC-LimnoTech Collaborative Work Plan during our workshop (March 24-25, 2020), expanding and automating the testing of HSP2 vs. HSPF is an immediate priority.
Our objective for testing is to ensure that HSP2 provides the same results as HSPF for:
- all HSP2 releases,
- several relevant operating systems and software environments, and
- a selected group of watershed models that have been calibrated and examined for real-world water management applications and that represent a range of watershed properties.
We decided that:
- HSPF “reference” model runs should be added to repo and considered static/stable
- HSP2 outputs will continually evolve, expanding as new process modules are implemented
- Comparisons will be point-by-point for major output time series, to "byte-precision" of about 3 significant figures to allow for rounding errors
- We can't do traditional unit testing of individual routines because HSPF doesn't save that data.
RESPEC has two test models to contribute:
- Test10
- Calleg
LimnoTech will add additional models:
- We selected 2 watersheds that we’ve recently modeled in HSPF, and selected a single sub-watershed (to simplify running)
- Grant River, MI. Relatively simple
- Zumbro River, MN. More complicated. Full water quality suite.
- Hydrological Response Unit (HRU) testing
- 5-10 micro watersheds (1 HRU + a few stream reaches)
Let's use this issue to track progress on all the smaller tasks required to complete this.
We have already added some reference models and testing code with 49c71f3, 60378a7, and LimnoTech@130bef2.
cc: @rheaphy, @PaulDudaRESPEC, @steveskrip, @ptomasula,