clean-up of aggregation features (#315)

* add `equals()` function (with tests) * refactor auxiliary aggregation functions * add check that arg in `equals` is an `IamDataFrame` (and test) * split `aggregate[_region]()` function into public and internal parts change return type to `IamDataFrame` * refactor aggregation tests to expect `IamDataFrame` as return object * use `equals()` in downscale-tests * move auxiliary functions related to aggregation features to own file * add region-column to `PRICE_MAX_DF` * smarter aggregation-tests parametrization * remove duplicate test * change order of tests * smarter parametrization and re-ordering of region-aggregation tests * rename full-feature dataframe for aggregation tests to `simple_df` * remove unnecessary imports * add treatment of empty result of aggregation (and tests) * change return object from `check_aggregate[_region]()` * fix return-object of aggregate-region with weights * complete rework of consistency-checking tutorial * update list of tutorials on doc-pages * fix docstrings * refactor region/subregions tests * refactor aggregate-and-append tests * remove duplicate "passing" tests (already covered by new tests) * refactor check-aggregate(-region) tests * refactor top-level `check_aggregate()` test (with `exclude_on_fail`) * clean-up * refactor tests for log-messages of region aggregation * remove tests that are duplicated by new test suite * refactor test for `check_internal_consistency()´ * remove test data for previous aggregation test suite * make `aggregating_with_negative_values` tutorial pass with changed API * appease stickler * add aggregation/downscaling tutorial data * add to release notes * add explicit aggregation tests for datetime time format * fix implementation of weighted average for datetime time format * fix `downscale_region()` for datetime time format * add `pyam.testing.assert_frame_equal()` (suggested by @gidden) * refactor functions for aggregate-module (suggested by @gidden) * add docstring (suggested by @gidden) following pandas.DataFrame.equals, see https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.equals.html * appease stickler * add `region` level to index of frame returned by `check_aggregate_region()` * change `check_internal_consistency` to return a concatenated dataframe * update API changes in release notes * update the tutorial * appease stickler * Minor updates in the notebooks * cleaner `import` in testing module Co-Authored-By: Zeb Nicholls <[email protected]> * refactor to `group_and_agg()` (suggested by @znicholls) * add continue in variable-components-loop (suggested by @znicholls) * fix hard-coded region in `check_aggregate_region()` (found by @znicholls) * allow `components` to be passed through `check_internal_consistency()` (per comment by @znicholls) * add parameter to `equals()` docstring * appease stickler * add API change of `check_internal_consistency()` to readme * Move aggregate stuff into 'private' API * appease stickler, revert to import functions instead of private module Co-authored-by: Zeb Nicholls <[email protected]>
IAMconsortium · Jan 13, 2020 · 412ff2a · 412ff2a
1 parent ba32927
commit 412ff2a
Show file tree

Hide file tree

Showing 14 changed files with 1,002 additions and 956 deletions.
diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md
@@ -6,8 +6,18 @@ PR [#305](https://github.com/IAMconsortium/pyam/pull/305) changed the default
 behaviour of `aggregate_region()` regarding the treatment of components at the
 region-level. To keep the previous behaviour, add `components=True`.
 
+PR [#315](https://github.com/IAMconsortium/pyam/pull/314) changed the return
+type of `aggregate[_region]()` to an `IamDataFrame` instance. 
+To keep the previous behaviour, add `timeseries()`.
+The object returned by `[check_]aggregate[_region]()` now includes both the
+actual and the expected value as a `pd.DataFrame` instance.
+The function `check_internal_consistency()` now returns a concatenated dataframe
+rather than a dictionary and also includes optional treatment of components
+(see paragraph above). To keep the previous behaviour, add `components=True`.
+
 ## Individual Updates
 
+- [#315](https://github.com/IAMconsortium/pyam/pull/315) Add `equals()` feature, change return types of `[check_]aggregate[_region]()`, rework aggregation tutorial
 - [#314](https://github.com/IAMconsortium/pyam/pull/314) Update IPCC color scheme colors and add SSP-only colors
 - [#313](https://github.com/IAMconsortium/pyam/pull/313) Add feature to `downscale` timeseries data to subregions using another variable as proxy
 - [#312](https://github.com/IAMconsortium/pyam/pull/312) Allow passing list of variables to `aggregate` functions

diff --git a/doc/source/tutorials.rst b/doc/source/tutorials.rst
@@ -8,8 +8,8 @@ Tutorials
    :maxdepth: 1
 
    tutorials/pyam_first_steps.ipynb
-   tutorials/aggregating_variables_and_plotting_with_negative_values.ipynb
+   tutorials/aggregating_downscaling_consistency.ipynb.ipynb
    tutorials/ipcc_colors.ipynb
-   tutorials/pyam_logo.ipynb
    tutorials/iiasa_dbs.ipynb
-   tutorials/checking_databases.ipynb
+   tutorials/aggregating_variables_and_plotting_with_negative_values.ipynb
+   tutorials/pyam_logo.ipynb