-
Notifications
You must be signed in to change notification settings - Fork 4
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #24 from willu47/will/environments
Review of workflow
- Loading branch information
Showing
1,516 changed files
with
1,840 additions
and
103 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,55 @@ | ||
# Temporary and binary files | ||
*~ | ||
*.py[cod] | ||
*.so | ||
*.cfg | ||
!.isort.cfg | ||
!setup.cfg | ||
*.orig | ||
*.log | ||
*.pot | ||
__pycache__/* | ||
.cache/* | ||
.*.swp | ||
*/.ipynb_checkpoints/* | ||
.DS_Store | ||
.ipynb_checkpoints/ | ||
.snakemake | ||
|
||
# Project files | ||
.ropeproject | ||
.project | ||
.pydevproject | ||
.settings | ||
.idea | ||
tags | ||
.vscode | ||
|
||
# Package files | ||
*.egg | ||
*.eggs/ | ||
.installed.cfg | ||
*.egg-info | ||
|
||
# Unittest and coverage | ||
htmlcov/* | ||
.coverage | ||
.tox | ||
junit.xml | ||
coverage.xml | ||
.pytest_cache/ | ||
|
||
# Build and docs folder/files | ||
|
||
# Per-project virtualenvs | ||
.venv*/ | ||
|
||
# Plots | ||
|
||
# Temporary workflow files | ||
working_directory/* | ||
|
||
# Outputs | ||
results/* | ||
*.png |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,35 +1,44 @@ | ||
# OSeMBE_ECEMF | ||
This repository contains the scenarios modelled in OSeMBE throughout the H2020 project ECEMF -European Climate and Energy Modelling Forum. And the repo contains a workflow to run OSeMOSYS models from datapackage to results file in IAMC format. | ||
# OSeMOSYS workflow | ||
This workflow allows to run one or multiple scenarios. Starting from an OSeMOSYS datapackage going through all steps running the model with gurobi and producing the results in IAMC format. | ||
|
||
In addition to the standard OSeMOSYS outputs, the workflow can produce csvs containing the dual values for all constrains in the model. | ||
This repository contains the scenarios modelled in OSeMBE throughout the H2020 project ECEMF - | ||
European Climate and Energy Modelling Forum. | ||
|
||
The repo contains a workflow to run OSeMOSYS models from datapackage to results file in IAMC format. | ||
|
||
## OSeMOSYS workflow | ||
|
||
This workflow allows to run one or multiple scenarios. | ||
Starting from an OSeMOSYS datapackage going through all steps, | ||
running the model with the Gurobi solver and producing the results in IAMC format. | ||
|
||
In addition to the standard OSeMOSYS outputs, the workflow can produce CSV files containing the dual values for all constraints in the model. | ||
|
||
### Installation | ||
|
||
## Installation | ||
Install snakemake using conda into a new environment called `snakemake`: | ||
|
||
```bash | ||
conda install -c conda-forge mamba | ||
mamba create -c bioconda -c conda-forge -n snakemake snakemake-minimal | ||
``` | ||
## Configuring the workflow | ||
### Input: | ||
|
||
- datapackage(s) | ||
The workflow manages dependencies through conda environments. | ||
Dependencies are defined per rule and are installed upon first running the workflow. | ||
|
||
### Output: | ||
### Configuring the workflow | ||
|
||
- xlsx-files with model results in IAMC format | ||
3. Place the script `resultify.py` from the repo [osemosys2iamc](https://github.com/OSeMOSYS/osemosys2iamc/tree/osembe) in the root folder of the project. | ||
|
||
### Usage: | ||
### Adding new scenarios | ||
|
||
1. Place datapackage(s) in the folder `input_data`. Each datapackage should be placed in a folder | ||
that is named after the scenario, e.g. `baseline`. | ||
4. Check that the file `config.yaml`, which defines the conversion of OSeMOSYS results to IAMC format | ||
is suitable for the model. | ||
|
||
### Running the workflow | ||
|
||
1. Place datapackage(s) in the folder `data`. Each datapackage should be in a folder that is named after the scenario, e.g. `baseline`. | ||
2. Open the snakefile in a text-editor of your choice. Replace the scenario names potentially already in the list of `SCENARIOS` in line 1 with the names of your scenarios. The line should look similar to the below: | ||
```bash | ||
SCENARIOS = ['baseline', 'scenario1']` | ||
``` | ||
3. Place the script `resultify.py` from the repo [osemosys2iamc](https://github.com/OSeMOSYS/osemosys2iamc/tree/osembe) in your folder. | ||
4. Make sure that the file `data/config.yaml`is suitable for your model. | ||
3. ***Optional***: To retrieve dual values from your model you need to edit the list of `constraints` in the file `run.py`. | ||
4. Open terminal or command prompt in the directory of the snakefile. | ||
4. Open terminal or command prompt and change to the directory of the snakefile. | ||
5. ***Optional***: Perform a dry run to test snakemake with the command: `snakemake -n` | ||
5. Start the scenario runs with the command `snakemake --cores <number of cores to be used> --use-conda` | ||
5. Start the scenario runs with the command `snakemake --cores <number of cores to be used> --use-conda` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,124 @@ | ||
data/ATReserveMarginTagTechnology.csv | ||
data/AccumulatedAnnualDemand.csv | ||
data/AnnualEmissionLimit.csv | ||
data/AnnualExogenousEmission.csv | ||
data/AvailabilityFactor.csv | ||
data/BEReserveMarginTagTechnology.csv | ||
data/BGReserveMarginTagTechnology.csv | ||
data/CHReserveMarginTagTechnology.csv | ||
data/CYReserveMarginTagTechnology.csv | ||
data/CZReserveMarginTagTechnology.csv | ||
data/CapacityFactor.csv | ||
data/CapacityOfOneTechnologyUnit.csv | ||
data/CapacityToActivityUnit.csv | ||
data/CapitalCost.csv | ||
data/CapitalCostStorage.csv | ||
data/Conversionld.csv | ||
data/Conversionlh.csv | ||
data/Conversionls.csv | ||
data/DAILYTIMEBRACKET.csv | ||
data/DAYTYPE.csv | ||
data/DEReserveMarginTagTechnology.csv | ||
data/DKReserveMarginTagTechnology.csv | ||
data/DaySplit.csv | ||
data/DaysInDayType.csv | ||
data/DepreciationMethod.csv | ||
data/DiscountRate.csv | ||
data/DiscountRateStorage.csv | ||
data/EEReserveMarginTagTechnology.csv | ||
data/EMISSION.csv | ||
data/ESReserveMarginTagTechnology.csv | ||
data/EmissionActivityRatio.csv | ||
data/EmissionsPenalty.csv | ||
data/FIReserveMarginTagTechnology.csv | ||
data/FRReserveMarginTagTechnology.csv | ||
data/FUEL.csv | ||
data/FixedCost.csv | ||
data/GRReserveMarginTagTechnology.csv | ||
data/HRReserveMarginTagTechnology.csv | ||
data/HUReserveMarginTagTechnology.csv | ||
data/IEReserveMarginTagTechnology.csv | ||
data/ITReserveMarginTagTechnology.csv | ||
data/InputActivityRatio.csv | ||
data/LTReserveMarginTagTechnology.csv | ||
data/LUReserveMarginTagTechnology.csv | ||
data/LVReserveMarginTagTechnology.csv | ||
data/MODE_OF_OPERATION.csv | ||
data/MTReserveMarginTagTechnology.csv | ||
data/MinStorageCharge.csv | ||
data/ModelPeriodEmissionLimit.csv | ||
data/ModelPeriodExogenousEmission.csv | ||
data/NLReserveMarginTagTechnology.csv | ||
data/NOReserveMarginTagTechnology.csv | ||
data/OperationalLife.csv | ||
data/OperationalLifeStorage.csv | ||
data/OutputActivityRatio.csv | ||
data/PLReserveMarginTagTechnology.csv | ||
data/PTReserveMarginTagTechnology.csv | ||
data/REGION.csv | ||
data/REMinProductionTarget.csv | ||
data/RETagFuel.csv | ||
data/RETagTechnology.csv | ||
data/ROReserveMarginTagTechnology.csv | ||
data/ReserveMargin.csv | ||
data/ReserveMarginTagFuel.csv | ||
data/ResidualCapacity.csv | ||
data/ResidualStorageCapacity.csv | ||
data/SEASON.csv | ||
data/SEReserveMarginTagTechnology.csv | ||
data/SIReserveMarginTagTechnology.csv | ||
data/SKReserveMarginTagTechnology.csv | ||
data/STORAGE.csv | ||
data/SpecifiedAnnualDemand.csv | ||
data/SpecifiedDemandProfile.csv | ||
data/StorageLevelStart.csv | ||
data/StorageMaxChargeRate.csv | ||
data/StorageMaxDischargeRate.csv | ||
data/TECHNOLOGY.csv | ||
data/TIMESLICE.csv | ||
data/TechnologyFromStorage.csv | ||
data/TechnologyToStorage.csv | ||
data/TotalAnnualMaxCapacity.csv | ||
data/TotalAnnualMaxCapacityInvestment.csv | ||
data/TotalAnnualMinCapacity.csv | ||
data/TotalAnnualMinCapacityInvestment.csv | ||
data/TotalTechnologyAnnualActivityLowerLimit.csv | ||
data/TotalTechnologyAnnualActivityUpperLimit.csv | ||
data/TotalTechnologyModelPeriodActivityLowerLimit.csv | ||
data/TotalTechnologyModelPeriodActivityUpperLimit.csv | ||
data/TradeRoute.csv | ||
data/UKReserveMarginTagTechnology.csv | ||
data/VariableCost.csv | ||
data/YEAR.csv | ||
data/YearSplit.csv | ||
data/YearlyPeakAT.csv | ||
data/YearlyPeakBE.csv | ||
data/YearlyPeakBG.csv | ||
data/YearlyPeakCH.csv | ||
data/YearlyPeakCY.csv | ||
data/YearlyPeakCZ.csv | ||
data/YearlyPeakDE.csv | ||
data/YearlyPeakDK.csv | ||
data/YearlyPeakEE.csv | ||
data/YearlyPeakES.csv | ||
data/YearlyPeakFI.csv | ||
data/YearlyPeakFR.csv | ||
data/YearlyPeakGR.csv | ||
data/YearlyPeakHR.csv | ||
data/YearlyPeakHU.csv | ||
data/YearlyPeakIE.csv | ||
data/YearlyPeakIT.csv | ||
data/YearlyPeakLT.csv | ||
data/YearlyPeakLU.csv | ||
data/YearlyPeakLV.csv | ||
data/YearlyPeakMT.csv | ||
data/YearlyPeakNL.csv | ||
data/YearlyPeakNO.csv | ||
data/YearlyPeakPL.csv | ||
data/YearlyPeakPT.csv | ||
data/YearlyPeakRO.csv | ||
data/YearlyPeakSE.csv | ||
data/YearlyPeakSI.csv | ||
data/YearlyPeakSK.csv | ||
data/YearlyPeakUK.csv | ||
data/default_values.csv |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
channels: | ||
- anaconda | ||
dependencies: | ||
- graphviz |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -9,4 +9,4 @@ dependencies: | |
- xlrd | ||
- pip | ||
- pip: | ||
- otoole | ||
- git+https://github.com/HauHe/otoole.git@osembe#egg=otoole |
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Oops, something went wrong.