-
Notifications
You must be signed in to change notification settings - Fork 6
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- v2.19.0
- v2.18.3
- v2.18.2
- v2.18.1
- v2.18.0
- v2.17.6
- v2.17.5
- v2.17.4
- v2.17.3
- v2.17.2
- v2.17.1
- v2.17.0
- v2.16.8
- v2.16.7
- v2.16.6
- v2.16.5
- v2.16.4
- v2.16.3
- v2.16.2
- v2.16.1
- v2.16.0
- v2.15.6
- v2.15.5
- v2.15.4
- v2.15.3
- v2.15.2
- v2.15.1
- v2.15.0
- v2.14.6
- v2.14.5
- v2.14.4
- v2.14.3
- v2.14.2
- v2.14.1
- v2.14.0
- v2.13.2
- v2.13.1
- v2.13.0
Showing
231 changed files
with
23,193 additions
and
3,503 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,46 @@ | ||
--- | ||
name: Bug report | ||
about: Create a report to help us improve | ||
title: '' | ||
labels: 'bug' | ||
assignees: '' | ||
|
||
--- | ||
|
||
## Description | ||
|
||
Provide a clear and concise description of the issue. | ||
## Steps to Reproduce | ||
|
||
### Steps to reproduce the behavior: | ||
1. Go to '...' | ||
2. Click on '....' | ||
3. Scroll down to '....' | ||
4. See error | ||
## Actual Behavior | ||
|
||
Describe what actually happened. | ||
## Expected Behavior | ||
|
||
Describe what you expected to happen. | ||
## Screenshots | ||
|
||
If applicable, add screenshots to help explain your problem. | ||
## Possible Solution | ||
|
||
If you have a solution in mind, describe it here. | ||
## Environment | ||
|
||
- [ ] Production | ||
- [ ] Staging | ||
## Additional Information | ||
### Add any other context about the problem here. | ||
|
||
- If applicable, provide the study ID that this issue relates to: | ||
- Any relevant logs or error messages: [e.g. console outputs, stack traces] | ||
- Any relevant code snippets: [e.g. relevant function calls, configuration files] | ||
- Any relevant library versions: [e.g. React 17.0.1] | ||
- The date and time when the issue occurred: | ||
- Browser and version: [e.g. Chrome 88, Safari 14] | ||
- Operating System: [e.g. Windows 10, macOS 11.2] | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
--- | ||
name: Feature request | ||
about: Suggest an idea for this project | ||
title: '' | ||
labels: 'feature request' | ||
assignees: '' | ||
|
||
--- | ||
|
||
## Problem Statement | ||
|
||
A concise statement that clearly defines the problem the feature is trying to solve, or user problem it addresses: | ||
|
||
## Proposed Solution | ||
|
||
A high-level description of the proposed solution, including: | ||
|
||
- The desired outcome | ||
|
||
- Screenshots or mockups that illustrate the desired outcome (if available) | ||
|
||
- Any specific requirements or constraints for the feature | ||
|
||
- Add any other context or examples to help explain the requested feature | ||
|
||
## Developer Notes | ||
|
||
- Any potential roadblocks or concerns: [e.g. dependencies on other features, limitations in technology] | ||
|
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,16 @@ | ||
__version__ = "2.12.2" | ||
""" | ||
Antares Web | ||
This module contains the project metadata. | ||
""" | ||
from pathlib import Path | ||
|
||
# Standard project metadata | ||
|
||
__version__ = "2.13.0" | ||
__author__ = "RTE, Antares Web Team" | ||
__date__ = "2023-03-09" | ||
# noinspection SpellCheckingInspection | ||
__credits__ = "(c) Réseau de Transport de l’Électricité (RTE)" | ||
|
||
ROOT_DIR: Path = Path(__file__).resolve().parent |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,17 +1,14 @@ | ||
from enum import Enum | ||
from typing import Optional | ||
|
||
from pydantic import BaseModel | ||
from sqlalchemy import Column, String, Integer, Boolean, DateTime, ForeignKey, Sequence # type: ignore | ||
from sqlalchemy.orm import relationship # type: ignore | ||
|
||
|
||
class MaintenanceMode(str, Enum): | ||
NORMAL_MODE = "NORMAL" | ||
MAINTENANCE_MODE = "MAINTENANCE" | ||
|
||
@staticmethod | ||
def to_str(element: bool) -> str: | ||
if element: | ||
return MaintenanceMode.MAINTENANCE_MODE.value | ||
return MaintenanceMode.NORMAL_MODE.value | ||
@classmethod | ||
def from_bool(cls, flag: bool) -> "MaintenanceMode": | ||
return {False: cls.NORMAL_MODE, True: cls.MAINTENANCE_MODE}[flag] | ||
|
||
def __bool__(self) -> bool: | ||
cls = self.__class__ | ||
return {cls.NORMAL_MODE: False, cls.MAINTENANCE_MODE: True}[self] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
394 changes: 223 additions & 171 deletions
394
antarest/launcher/adapters/slurm_launcher/slurm_launcher.py
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,115 @@ | ||
from enum import Enum | ||
from pathlib import Path, PurePosixPath | ||
from typing import Union, Optional, Dict, TypedDict, Any, List | ||
|
||
from pydantic import StrictFloat, StrictInt, StrictStr, StrictBool | ||
|
||
from antarest.study.business.utils import ( | ||
execute_or_add_commands, | ||
FormFieldsBaseModel, | ||
FieldInfo, | ||
) | ||
from antarest.study.model import Study | ||
from antarest.study.storage.storage_service import StudyStorageService | ||
from antarest.study.storage.variantstudy.model.command.update_config import ( | ||
UpdateConfig, | ||
) | ||
|
||
|
||
class TimeSeriesInterpretation(str, Enum): | ||
POWER_GENERATION = "power-generation" | ||
PRODUCTION_FACTOR = "production-factor" | ||
|
||
|
||
RENEWABLE_PATH = "input/renewables/clusters/{area}/list/{cluster}" | ||
|
||
|
||
class RenewableFormFields(FormFieldsBaseModel): | ||
group: Optional[StrictStr] | ||
name: Optional[StrictStr] | ||
ts_interpretation: Optional[TimeSeriesInterpretation] | ||
unit_count: Optional[StrictInt] | ||
enabled: Optional[StrictBool] | ||
nominal_capacity: Optional[StrictInt] | ||
|
||
|
||
FIELDS_INFO: Dict[str, FieldInfo] = { | ||
"group": { | ||
"path": f"{RENEWABLE_PATH}/group", | ||
"default_value": "", | ||
}, | ||
"name": { | ||
"path": f"{RENEWABLE_PATH}/name", | ||
"default_value": "", | ||
}, | ||
"ts_interpretation": { | ||
"path": f"{RENEWABLE_PATH}/ts-interpretation", | ||
"default_value": TimeSeriesInterpretation.POWER_GENERATION.value, | ||
}, | ||
"unit_count": { | ||
"path": f"{RENEWABLE_PATH}/unitcount", | ||
"default_value": 0, | ||
}, | ||
"enabled": { | ||
"path": f"{RENEWABLE_PATH}/enabled", | ||
"default_value": True, | ||
}, | ||
"nominal_capacity": { | ||
"path": f"{RENEWABLE_PATH}/nominalcapacity", | ||
"default_value": 0, | ||
}, | ||
} | ||
|
||
|
||
def format_path(path: str, area_id: str, cluster_id: str) -> str: | ||
return path.format(area=area_id, cluster=cluster_id) | ||
|
||
|
||
class RenewableManager: | ||
def __init__(self, storage_service: StudyStorageService): | ||
self.storage_service = storage_service | ||
|
||
def get_field_values( | ||
self, study: Study, area_id: str, cluster_id: str | ||
) -> RenewableFormFields: | ||
file_study = self.storage_service.get_storage(study).get_raw(study) | ||
renewable_config = file_study.tree.get( | ||
format_path(RENEWABLE_PATH, area_id, cluster_id).split("/") | ||
) | ||
|
||
def get_value(field_info: FieldInfo) -> Any: | ||
target_name = PurePosixPath(field_info["path"]).name | ||
return renewable_config.get( | ||
target_name, field_info["default_value"] | ||
) | ||
|
||
return RenewableFormFields.construct( | ||
**{name: get_value(info) for name, info in FIELDS_INFO.items()} | ||
) | ||
|
||
def set_field_values( | ||
self, | ||
study: Study, | ||
area_id: str, | ||
cluster_id: str, | ||
field_values: RenewableFormFields, | ||
) -> None: | ||
commands: List[UpdateConfig] = [] | ||
|
||
for field_name, value in field_values.__iter__(): | ||
if value is not None: | ||
info = FIELDS_INFO[field_name] | ||
|
||
commands.append( | ||
UpdateConfig( | ||
target=format_path(info["path"], area_id, cluster_id), | ||
data=value, | ||
command_context=self.storage_service.variant_study_service.command_factory.command_context, | ||
) | ||
) | ||
|
||
if commands: | ||
file_study = self.storage_service.get_storage(study).get_raw(study) | ||
execute_or_add_commands( | ||
study, file_study, commands, self.storage_service | ||
) |
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,194 @@ | ||
from enum import Enum | ||
from pathlib import PurePosixPath | ||
from typing import Optional, Dict, Any, List | ||
|
||
from pydantic import StrictStr, StrictBool | ||
|
||
from antarest.study.business.utils import ( | ||
FormFieldsBaseModel, | ||
FieldInfo, | ||
execute_or_add_commands, | ||
) | ||
from antarest.study.model import Study | ||
from antarest.study.storage.storage_service import StudyStorageService | ||
from antarest.study.storage.variantstudy.model.command.update_config import ( | ||
UpdateConfig, | ||
) | ||
|
||
|
||
class TimeSeriesGenerationOption(str, Enum): | ||
USE_GLOBAL_PARAMETER = "use global parameter" | ||
FORCE_NO_GENERATION = "force no generation" | ||
FORCE_GENERATION = "force generation" | ||
|
||
|
||
class LawOption(str, Enum): | ||
UNIFORM = "uniform" | ||
GEOMETRIC = "geometric" | ||
|
||
|
||
THERMAL_PATH = "input/thermal/clusters/{area}/list/{cluster}" | ||
|
||
|
||
class ThermalFormFields(FormFieldsBaseModel): | ||
group: Optional[StrictStr] | ||
name: Optional[StrictStr] | ||
unit_count: Optional[int] | ||
enabled: Optional[StrictBool] | ||
nominal_capacity: Optional[int] | ||
gen_ts: Optional[TimeSeriesGenerationOption] | ||
min_stable_power: Optional[int] | ||
min_up_time: Optional[int] | ||
min_down_time: Optional[int] | ||
must_run: Optional[StrictBool] | ||
spinning: Optional[int] | ||
co2: Optional[int] | ||
volatility_forced: Optional[int] | ||
volatility_planned: Optional[int] | ||
law_forced: Optional[LawOption] | ||
law_planned: Optional[LawOption] | ||
marginal_cost: Optional[int] | ||
spread_cost: Optional[int] | ||
fixed_cost: Optional[int] | ||
startup_cost: Optional[int] | ||
market_bid_cost: Optional[int] | ||
|
||
|
||
FIELDS_INFO: Dict[str, FieldInfo] = { | ||
"group": { | ||
"path": f"{THERMAL_PATH}/group", | ||
"default_value": "", | ||
}, | ||
"name": { | ||
"path": f"{THERMAL_PATH}/name", | ||
"default_value": "", | ||
}, | ||
"unit_count": { | ||
"path": f"{THERMAL_PATH}/unitcount", | ||
"default_value": 0, | ||
}, | ||
"enabled": { | ||
"path": f"{THERMAL_PATH}/enabled", | ||
"default_value": True, | ||
}, | ||
"nominal_capacity": { | ||
"path": f"{THERMAL_PATH}/nominalcapacity", | ||
"default_value": 0, | ||
}, | ||
"gen_ts": { | ||
"path": f"{THERMAL_PATH}/gen-ts", | ||
"default_value": TimeSeriesGenerationOption.USE_GLOBAL_PARAMETER.value, | ||
}, | ||
"min_stable_power": { | ||
"path": f"{THERMAL_PATH}/min-stable-power", | ||
"default_value": 0, | ||
}, | ||
"min_up_time": { | ||
"path": f"{THERMAL_PATH}/min-up-time", | ||
"default_value": 1, | ||
}, | ||
"min_down_time": { | ||
"path": f"{THERMAL_PATH}/min-down-time", | ||
"default_value": 1, | ||
}, | ||
"must_run": { | ||
"path": f"{THERMAL_PATH}/must-run", | ||
"default_value": False, | ||
}, | ||
"spinning": { | ||
"path": f"{THERMAL_PATH}/spinning", | ||
"default_value": 0, | ||
}, | ||
"co2": { | ||
"path": f"{THERMAL_PATH}/co2", | ||
"default_value": 0, | ||
}, | ||
"volatility_forced": { | ||
"path": f"{THERMAL_PATH}/volatility.forced", | ||
"default_value": 0, | ||
}, | ||
"volatility_planned": { | ||
"path": f"{THERMAL_PATH}/volatility.planned", | ||
"default_value": 0, | ||
}, | ||
"law_forced": { | ||
"path": f"{THERMAL_PATH}/law.forced", | ||
"default_value": LawOption.UNIFORM.value, | ||
}, | ||
"law_planned": { | ||
"path": f"{THERMAL_PATH}/law.planned", | ||
"default_value": LawOption.UNIFORM.value, | ||
}, | ||
"marginal_cost": { | ||
"path": f"{THERMAL_PATH}/marginal-cost", | ||
"default_value": 0, | ||
}, | ||
"spread_cost": { | ||
"path": f"{THERMAL_PATH}/spread-cost", | ||
"default_value": 0, | ||
}, | ||
"fixed_cost": { | ||
"path": f"{THERMAL_PATH}/fixed-cost", | ||
"default_value": 0, | ||
}, | ||
"startup_cost": { | ||
"path": f"{THERMAL_PATH}/startup-cost", | ||
"default_value": 0, | ||
}, | ||
"market_bid_cost": { | ||
"path": f"{THERMAL_PATH}/market-bid-cost", | ||
"default_value": 0, | ||
}, | ||
} | ||
|
||
|
||
def format_path(path: str, area_id: str, cluster_id: str) -> str: | ||
return path.format(area=area_id, cluster=cluster_id) | ||
|
||
|
||
class ThermalManager: | ||
def __init__(self, storage_service: StudyStorageService): | ||
self.storage_service = storage_service | ||
|
||
def get_field_values( | ||
self, study: Study, area_id: str, cluster_id: str | ||
) -> ThermalFormFields: | ||
file_study = self.storage_service.get_storage(study).get_raw(study) | ||
thermal_config = file_study.tree.get( | ||
format_path(THERMAL_PATH, area_id, cluster_id).split("/") | ||
) | ||
|
||
def get_value(field_info: FieldInfo) -> Any: | ||
target_name = PurePosixPath(field_info["path"]).name | ||
return thermal_config.get(target_name, field_info["default_value"]) | ||
|
||
return ThermalFormFields.construct( | ||
**{name: get_value(info) for name, info in FIELDS_INFO.items()} | ||
) | ||
|
||
def set_field_values( | ||
self, | ||
study: Study, | ||
area_id: str, | ||
cluster_id: str, | ||
field_values: ThermalFormFields, | ||
) -> None: | ||
commands: List[UpdateConfig] = [] | ||
|
||
for field_name, value in field_values.__iter__(): | ||
if value is not None: | ||
info = FIELDS_INFO[field_name] | ||
|
||
commands.append( | ||
UpdateConfig( | ||
target=format_path(info["path"], area_id, cluster_id), | ||
data=value, | ||
command_context=self.storage_service.variant_study_service.command_factory.command_context, | ||
) | ||
) | ||
|
||
if commands: | ||
file_study = self.storage_service.get_storage(study).get_raw(study) | ||
execute_or_add_commands( | ||
study, file_study, commands, self.storage_service | ||
) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,71 @@ | ||
import configparser | ||
from typing import Any, Dict, Union, Optional | ||
|
||
|
||
def _convert_value(value: Any) -> str: | ||
""" | ||
Convert a value to a string using the specific format of Antares INI files. | ||
- strings are preserved, | ||
- ``None`` is converted to an empty string, | ||
- booleans are converted to "true"/"false" in lower case. | ||
- numbers are converted to strings using the :class:`str` function. | ||
""" | ||
if value is None: | ||
return "" | ||
elif value is True or value is False: | ||
return str(value).lower() | ||
else: | ||
return str(value) | ||
|
||
|
||
class AntaresSectionProxy(configparser.SectionProxy): | ||
""" | ||
This class extends the :class:`configparser.SectionProxy` class in order | ||
to store strings or other types of values by converting them according | ||
to the rules of Antares INI files. | ||
""" | ||
|
||
def __repr__(self) -> str: | ||
"""String representation of the section proxy used for debug.""" | ||
cls = self.__class__.__name__ | ||
return f"<{cls}: {self._name}>" | ||
|
||
def __setitem__(self, key: str, value: Any) -> None: | ||
""" | ||
Sets the value of the specified key in the section. | ||
""" | ||
super().__setitem__(key, _convert_value(value)) | ||
|
||
|
||
class AntaresConfigParser(configparser.RawConfigParser): | ||
""" | ||
This class extends the :class:`configparser.RawConfigParser` class in order | ||
to store strings or other types of values by converting them according | ||
to the rules of Antares INI files. | ||
""" | ||
|
||
_proxies: Dict[str, AntaresSectionProxy] | ||
_sections: Dict[str, Optional[Union[bool, int, float, str]]] | ||
|
||
def __init__(self, *args, **kwargs) -> None: # type: ignore | ||
super().__init__(*args, **kwargs) | ||
self._proxies[self.default_section] = AntaresSectionProxy( | ||
self, self.default_section | ||
) | ||
|
||
def add_section(self, section: str) -> None: | ||
super().add_section(section) | ||
self._proxies[section] = AntaresSectionProxy(self, section) | ||
|
||
def set(self, section: str, option: str, value: Any = None) -> None: | ||
super().set(section, option, _convert_value(value)) | ||
|
||
def _read(self, fp, fpname): # type: ignore | ||
# noinspection PyProtectedMember | ||
super()._read(fp, fpname) # type: ignore | ||
# cast section proxies to AntaresSectionProxy | ||
proxies = self._proxies | ||
for name, proxy in self._sections.items(): | ||
if not isinstance(proxy, AntaresSectionProxy): | ||
proxies[name] = AntaresSectionProxy(self, name) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
42 changes: 42 additions & 0 deletions
42
antarest/study/storage/rawstudy/model/filesystem/config/exceptions.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,42 @@ | ||
from pathlib import Path | ||
from typing import cast | ||
|
||
|
||
class BaseConfigError(Exception): | ||
"""Base class of the configuration errors.""" | ||
|
||
|
||
class SimulationParsingError(BaseConfigError): | ||
def __init__(self, output_path: Path, reason: str): | ||
super().__init__(output_path, reason) | ||
|
||
@property | ||
def output_path(self) -> Path: | ||
return cast(Path, self.args[0]) | ||
|
||
@property | ||
def reason(self) -> str: | ||
return cast(str, self.args[1]) | ||
|
||
def __str__(self) -> str: | ||
output_path = self.output_path | ||
reason = self.reason | ||
return f"Fail to parse the simulation file '{output_path}': {reason}" | ||
|
||
|
||
class XpansionParsingError(BaseConfigError): | ||
def __init__(self, xpansion_json: Path, reason: str): | ||
super().__init__(xpansion_json, reason) | ||
|
||
@property | ||
def xpansion_json(self) -> Path: | ||
return cast(Path, self.args[0]) | ||
|
||
@property | ||
def reason(self) -> str: | ||
return cast(str, self.args[1]) | ||
|
||
def __str__(self) -> str: | ||
xpansion_json = self.xpansion_json | ||
reason = self.reason | ||
return f"Fail to parse the Xpansion file '{xpansion_json}': {reason}" |
711 changes: 354 additions & 357 deletions
711
antarest/study/storage/rawstudy/model/filesystem/config/files.py
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,195 @@ | ||
from http import HTTPStatus | ||
from http.client import HTTPException | ||
from pathlib import Path | ||
from typing import NamedTuple, Callable | ||
import logging | ||
import re | ||
import shutil | ||
import tempfile | ||
import time | ||
|
||
from antarest.core.exceptions import StudyValidationError | ||
|
||
from .upgrader_710 import upgrade_710 | ||
from .upgrader_720 import upgrade_720 | ||
from .upgrader_800 import upgrade_800 | ||
from .upgrader_810 import upgrade_810 | ||
from .upgrader_820 import upgrade_820 | ||
from .upgrader_830 import upgrade_830 | ||
from .upgrader_840 import upgrade_840 | ||
from .upgrader_850 import upgrade_850 | ||
|
||
|
||
logger = logging.getLogger(__name__) | ||
|
||
|
||
class UpgradeMethod(NamedTuple): | ||
"""Raw study upgrade method (old version, new version, upgrade function).""" | ||
|
||
old: str | ||
new: str | ||
method: Callable[[Path], None] | ||
|
||
|
||
UPGRADE_METHODS = [ | ||
UpgradeMethod("700", "710", upgrade_710), | ||
UpgradeMethod("710", "720", upgrade_720), | ||
UpgradeMethod("720", "800", upgrade_800), | ||
UpgradeMethod("800", "810", upgrade_810), | ||
UpgradeMethod("810", "820", upgrade_820), | ||
UpgradeMethod("820", "830", upgrade_830), | ||
UpgradeMethod("830", "840", upgrade_840), | ||
UpgradeMethod("840", "850", upgrade_850), | ||
] | ||
|
||
|
||
class InvalidUpgrade(HTTPException): | ||
def __init__(self, message: str) -> None: | ||
super().__init__(HTTPStatus.UNPROCESSABLE_ENTITY, message) | ||
|
||
|
||
def find_next_version(from_version: str) -> str: | ||
""" | ||
Find the next study version from the given version. | ||
Args: | ||
from_version: The current version as a string. | ||
Returns: | ||
The next version as a string. | ||
If no next version was found, returns an empty string. | ||
""" | ||
return next( | ||
(meth.new for meth in UPGRADE_METHODS if from_version == meth.old), | ||
"", | ||
) | ||
|
||
|
||
def upgrade_study(study_path: Path, target_version: str) -> None: | ||
tmp_dir = Path( | ||
tempfile.mkdtemp( | ||
suffix=".upgrade.tmp", prefix="~", dir=study_path.parent | ||
) | ||
) | ||
shutil.copytree(study_path, tmp_dir, dirs_exist_ok=True) | ||
try: | ||
src_version = get_current_version(tmp_dir) | ||
can_upgrade_version(src_version, target_version) | ||
_do_upgrade(tmp_dir, src_version, target_version) | ||
except (StudyValidationError, InvalidUpgrade) as e: | ||
shutil.rmtree(tmp_dir) | ||
logger.warning(str(e)) | ||
raise | ||
except Exception as e: | ||
shutil.rmtree(tmp_dir) | ||
logger.error(f"Unhandled exception : {e}", exc_info=True) | ||
raise | ||
else: | ||
backup_dir = Path( | ||
tempfile.mkdtemp( | ||
suffix=".backup.tmp", prefix="~", dir=study_path.parent | ||
) | ||
) | ||
backup_dir.rmdir() | ||
study_path.rename(backup_dir) | ||
tmp_dir.rename(study_path) | ||
shutil.rmtree(backup_dir, ignore_errors=True) | ||
|
||
|
||
def get_current_version(study_path: Path) -> str: | ||
""" | ||
Get the current version of a study. | ||
Args: | ||
study_path: Path to the study. | ||
Returns: | ||
The current version of the study. | ||
Raises: | ||
StudyValidationError: If the version number is not found in the | ||
`study.antares` file or does not match the expected format. | ||
""" | ||
|
||
antares_path = study_path / "study.antares" | ||
pattern = r"version\s*=\s*([\w.-]+)\s*" | ||
with antares_path.open(encoding="utf-8") as lines: | ||
for line in lines: | ||
if match := re.fullmatch(pattern, line): | ||
return match[1].rstrip() | ||
raise StudyValidationError( | ||
f"File parsing error: the version number is not found in '{antares_path}'" | ||
f" or does not match the expected '{pattern}' format." | ||
) | ||
|
||
|
||
def can_upgrade_version(from_version: str, to_version: str) -> None: | ||
""" | ||
Checks if upgrading from one version to another is possible. | ||
Args: | ||
from_version: The current version of the study. | ||
to_version: The target version of the study. | ||
Raises: | ||
InvalidUpgrade: If the upgrade is not possible. | ||
""" | ||
if from_version == to_version: | ||
raise InvalidUpgrade( | ||
f"Your study is already in version '{to_version}'" | ||
) | ||
|
||
sources = [u.old for u in UPGRADE_METHODS] | ||
if from_version not in sources: | ||
raise InvalidUpgrade( | ||
f"Version '{from_version}' unknown: possible versions are {', '.join(sources)}" | ||
) | ||
|
||
targets = [u.new for u in UPGRADE_METHODS] | ||
if to_version not in targets: | ||
raise InvalidUpgrade( | ||
f"Version '{to_version}' unknown: possible versions are {', '.join(targets)}" | ||
) | ||
|
||
curr_version = from_version | ||
for src, dst in zip(sources, targets): | ||
if curr_version == src: | ||
curr_version = dst | ||
if curr_version == to_version: | ||
return | ||
|
||
# This code must be unreachable! | ||
raise InvalidUpgrade( | ||
f"Impossible to upgrade from version '{from_version}'" | ||
f" to version '{to_version}':" | ||
f" missing value in `UPGRADE_METHODS`." | ||
) | ||
|
||
|
||
def _update_study_antares_file(target_version: str, study_path: Path) -> None: | ||
file = study_path / "study.antares" | ||
content = file.read_text(encoding="utf-8") | ||
content = re.sub( | ||
r"^version\s*=.*$", | ||
f"version = {target_version}", | ||
content, | ||
flags=re.MULTILINE, | ||
) | ||
content = re.sub( | ||
r"^lastsave\s*=.*$", | ||
f"lastsave = {int(time.time())}", | ||
content, | ||
flags=re.MULTILINE, | ||
) | ||
file.write_text(content, encoding="utf-8") | ||
|
||
|
||
def _do_upgrade( | ||
study_path: Path, src_version: str, target_version: str | ||
) -> None: | ||
_update_study_antares_file(target_version, study_path) | ||
curr_version = src_version | ||
for old, new, method in UPGRADE_METHODS: | ||
if curr_version == old and curr_version != target_version: | ||
method(study_path) | ||
curr_version = new |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
from pathlib import Path | ||
from antarest.study.storage.rawstudy.io.reader import MultipleSameKeysIniReader | ||
from antarest.study.storage.rawstudy.io.writer.ini_writer import IniWriter | ||
from antarest.study.storage.rawstudy.model.filesystem.root.settings.generaldata import ( | ||
DUPLICATE_KEYS, | ||
) | ||
|
||
GENERAL_DATA_PATH = "settings/generaldata.ini" | ||
|
||
|
||
def upgrade_710(study_path: Path) -> None: | ||
""" | ||
Upgrade the study configuration to version 710. | ||
NOTE: | ||
The file `study.antares` is not upgraded here. | ||
Args: | ||
study_path: path to the study directory. | ||
""" | ||
|
||
reader = MultipleSameKeysIniReader(DUPLICATE_KEYS) | ||
data = reader.read(study_path / GENERAL_DATA_PATH) | ||
data["general"]["geographic-trimming"] = data["general"]["filtering"] | ||
data["general"]["thematic-trimming"] = False | ||
data["optimization"]["link-type"] = "local" | ||
data["other preferences"]["hydro-pricing-mode"] = "fast" | ||
del data["general"]["filtering"] | ||
writer = IniWriter(special_keys=DUPLICATE_KEYS) | ||
writer.write(data, study_path / GENERAL_DATA_PATH) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
from pathlib import Path | ||
|
||
|
||
def upgrade_720(study_path: Path) -> None: | ||
# There is no input modification between the 7.1.0 and the 7.2.0 version | ||
pass |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,34 @@ | ||
from pathlib import Path | ||
from antarest.study.storage.rawstudy.io.reader import MultipleSameKeysIniReader | ||
from antarest.study.storage.rawstudy.io.writer.ini_writer import IniWriter | ||
from antarest.study.storage.rawstudy.model.filesystem.root.settings.generaldata import ( | ||
DUPLICATE_KEYS, | ||
) | ||
|
||
GENERAL_DATA_PATH = "settings/generaldata.ini" | ||
|
||
|
||
def upgrade_800(study_path: Path) -> None: | ||
""" | ||
Upgrade the study configuration to version 800. | ||
NOTE: | ||
The file `study.antares` is not upgraded here. | ||
Args: | ||
study_path: path to the study directory. | ||
""" | ||
|
||
reader = MultipleSameKeysIniReader(DUPLICATE_KEYS) | ||
data = reader.read(study_path / GENERAL_DATA_PATH) | ||
data["other preferences"][ | ||
"hydro-heuristic-policy" | ||
] = "accommodate rule curves" | ||
data["optimization"]["include-exportstructure"] = False | ||
data["optimization"][ | ||
"include-unfeasible-problem-behavior" | ||
] = "error-verbose" | ||
data["general"]["custom-scenario"] = data["general"]["custom-ts-numbers"] | ||
del data["general"]["custom-ts-numbers"] | ||
writer = IniWriter(special_keys=DUPLICATE_KEYS) | ||
writer.write(data, study_path / GENERAL_DATA_PATH) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,28 @@ | ||
from pathlib import Path | ||
from antarest.study.storage.rawstudy.io.reader import MultipleSameKeysIniReader | ||
from antarest.study.storage.rawstudy.io.writer.ini_writer import IniWriter | ||
from antarest.study.storage.rawstudy.model.filesystem.root.settings.generaldata import ( | ||
DUPLICATE_KEYS, | ||
) | ||
|
||
GENERAL_DATA_PATH = "settings/generaldata.ini" | ||
|
||
|
||
def upgrade_810(study_path: Path) -> None: | ||
""" | ||
Upgrade the study configuration to version 810. | ||
NOTE: | ||
The file `study.antares` is not upgraded here. | ||
Args: | ||
study_path: path to the study directory. | ||
""" | ||
|
||
reader = MultipleSameKeysIniReader(DUPLICATE_KEYS) | ||
data = reader.read(study_path / GENERAL_DATA_PATH) | ||
data["other preferences"]["renewable-generation-modelling"] = "aggregated" | ||
writer = IniWriter(special_keys=DUPLICATE_KEYS) | ||
writer.write(data, study_path / GENERAL_DATA_PATH) | ||
study_path.joinpath("input", "renewables", "clusters").mkdir(parents=True) | ||
study_path.joinpath("input", "renewables", "series").mkdir(parents=True) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,50 @@ | ||
import glob | ||
from pathlib import Path | ||
|
||
import numpy | ||
import pandas # type: ignore | ||
|
||
|
||
def upgrade_820(study_path: Path) -> None: | ||
""" | ||
Upgrade the study configuration to version 820. | ||
NOTE: | ||
The file `study.antares` is not upgraded here. | ||
Args: | ||
study_path: path to the study directory. | ||
""" | ||
|
||
links = glob.glob(str(study_path / "input" / "links" / "*")) | ||
if len(links) > 0: | ||
for folder in links: | ||
folder_path = Path(folder) | ||
all_txt = glob.glob(str(folder_path / "*.txt")) | ||
if len(all_txt) > 0: | ||
(folder_path / "capacities").mkdir() | ||
for txt in all_txt: | ||
df = pandas.read_csv(txt, sep="\t", header=None) | ||
df_parameters = df.iloc[:, 2:8] | ||
df_direct = df.iloc[:, 0] | ||
df_indirect = df.iloc[:, 1] | ||
name = Path(txt).stem | ||
numpy.savetxt( | ||
folder_path / f"{name}_parameters.txt", | ||
df_parameters.values, | ||
delimiter="\t", | ||
fmt="%.6f", | ||
) | ||
numpy.savetxt( | ||
folder_path / "capacities" / f"{name}_direct.txt", | ||
df_direct.values, | ||
delimiter="\t", | ||
fmt="%.6f", | ||
) | ||
numpy.savetxt( | ||
folder_path / "capacities" / f"{name}_indirect.txt", | ||
df_indirect.values, | ||
delimiter="\t", | ||
fmt="%.6f", | ||
) | ||
(folder_path / f"{name}.txt").unlink() |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,42 @@ | ||
import glob | ||
from pathlib import Path | ||
|
||
from antarest.study.storage.rawstudy.io.reader import MultipleSameKeysIniReader | ||
from antarest.study.storage.rawstudy.io.writer.ini_writer import IniWriter | ||
from antarest.study.storage.rawstudy.model.filesystem.root.settings.generaldata import ( | ||
DUPLICATE_KEYS, | ||
) | ||
|
||
GENERAL_DATA_PATH = "settings/generaldata.ini" | ||
|
||
|
||
def upgrade_830(study_path: Path) -> None: | ||
""" | ||
Upgrade the study configuration to version 830. | ||
NOTE: | ||
The file `study.antares` is not upgraded here. | ||
Args: | ||
study_path: path to the study directory. | ||
""" | ||
|
||
reader = MultipleSameKeysIniReader(DUPLICATE_KEYS) | ||
data = reader.read(study_path / GENERAL_DATA_PATH) | ||
data["adequacy patch"] = { | ||
"include-adq-patch": False, | ||
"set-to-null-ntc-between-physical-out-for-first-step": True, | ||
"set-to-null-ntc-from-physical-out-to-physical-in-for-first-step": True, | ||
} | ||
data["optimization"]["include-split-exported-mps"] = False | ||
writer = IniWriter(special_keys=DUPLICATE_KEYS) | ||
writer.write(data, study_path / GENERAL_DATA_PATH) | ||
areas = glob.glob(str(study_path / "input" / "areas" / "*")) | ||
for folder in areas: | ||
folder_path = Path(folder) | ||
if folder_path.is_dir(): | ||
writer = IniWriter(special_keys=DUPLICATE_KEYS) | ||
writer.write( | ||
{"adequacy-patch": {"adequacy-patch-mode": "outside"}}, | ||
folder_path / "adequacy_patch.ini", | ||
) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
from pathlib import Path | ||
from antarest.study.storage.rawstudy.io.reader import MultipleSameKeysIniReader | ||
from antarest.study.storage.rawstudy.io.writer.ini_writer import IniWriter | ||
from antarest.study.storage.rawstudy.model.filesystem.root.settings.generaldata import ( | ||
DUPLICATE_KEYS, | ||
) | ||
|
||
GENERAL_DATA_PATH = "settings/generaldata.ini" | ||
MAPPING_TRANSMISSION_CAPACITIES = { | ||
True: "local-values", | ||
False: "null-for-all-links", | ||
"infinite": "infinite-for-all-links", | ||
} | ||
|
||
|
||
def upgrade_840(study_path: Path) -> None: | ||
""" | ||
Upgrade the study configuration to version 840. | ||
NOTE: | ||
The file `study.antares` is not upgraded here. | ||
Args: | ||
study_path: path to the study directory. | ||
""" | ||
|
||
reader = MultipleSameKeysIniReader(DUPLICATE_KEYS) | ||
data = reader.read(study_path / GENERAL_DATA_PATH) | ||
data["optimization"][ | ||
"transmission-capacities" | ||
] = MAPPING_TRANSMISSION_CAPACITIES[ | ||
data["optimization"]["transmission-capacities"] | ||
] | ||
del data["optimization"]["include-split-exported-mps"] | ||
writer = IniWriter(special_keys=DUPLICATE_KEYS) | ||
writer.write(data, study_path / GENERAL_DATA_PATH) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,34 @@ | ||
from pathlib import Path | ||
from antarest.study.storage.rawstudy.io.reader import MultipleSameKeysIniReader | ||
from antarest.study.storage.rawstudy.io.writer.ini_writer import IniWriter | ||
from antarest.study.storage.rawstudy.model.filesystem.root.settings.generaldata import ( | ||
DUPLICATE_KEYS, | ||
) | ||
|
||
# noinspection SpellCheckingInspection | ||
GENERAL_DATA_PATH = "settings/generaldata.ini" | ||
|
||
|
||
def upgrade_850(study_path: Path) -> None: | ||
""" | ||
Upgrade the study configuration to version 850. | ||
NOTE: | ||
The file `study.antares` is not upgraded here. | ||
Args: | ||
study_path: path to the study directory. | ||
""" | ||
|
||
reader = MultipleSameKeysIniReader(DUPLICATE_KEYS) | ||
data = reader.read(study_path / GENERAL_DATA_PATH) | ||
# fmt: off | ||
data["adequacy patch"]["price-taking-order"] = "DENS" | ||
data["adequacy patch"]["include-hurdle-cost-csr"] = False | ||
data["adequacy patch"]["check-csr-cost-function"] = False | ||
data["adequacy patch"]["threshold-initiate-curtailment-sharing-rule"] = 0.0 | ||
data["adequacy patch"]["threshold-display-local-matching-rule-violations"] = 0.0 | ||
data["adequacy patch"]["threshold-csr-variable-bounds-relaxation"] = 3 | ||
# fmt: on | ||
writer = IniWriter(special_keys=DUPLICATE_KEYS) | ||
writer.write(data, study_path / GENERAL_DATA_PATH) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,358 +0,0 @@ | ||
import glob | ||
import json | ||
import os | ||
import typing | ||
from datetime import datetime | ||
from http import HTTPStatus | ||
from http.client import HTTPException | ||
from pathlib import Path | ||
from typing import Optional | ||
|
||
import numpy | ||
import pandas # type: ignore | ||
|
||
from antarest.core.exceptions import StudyValidationError | ||
from antarest.study.storage.rawstudy.io.reader import MultipleSameKeysIniReader | ||
from antarest.study.storage.rawstudy.io.writer.ini_writer import IniWriter | ||
from antarest.study.storage.rawstudy.model.filesystem.root.settings.generaldata import ( | ||
DUPLICATE_KEYS, | ||
) | ||
|
||
|
||
def modify_file( | ||
study_path: str, | ||
file_path: str, | ||
key: str, | ||
parameter_to_add: Optional[str], | ||
value: typing.Any, | ||
parameter_to_delete: Optional[str], | ||
) -> None: | ||
reader = MultipleSameKeysIniReader(DUPLICATE_KEYS) | ||
file = glob.glob(os.path.join(study_path, file_path))[0] | ||
path = Path(file) | ||
data = reader.read(path) | ||
if key in data: | ||
if parameter_to_add is not None: | ||
data[key][parameter_to_add] = value | ||
if parameter_to_delete is not None: | ||
del data[key][parameter_to_delete] | ||
else: | ||
if parameter_to_add is not None: | ||
data[key] = {parameter_to_add: value} | ||
writer = IniWriter(special_keys=DUPLICATE_KEYS) | ||
writer.write(data, path) | ||
|
||
|
||
def find_value_in_file( | ||
study_path: str, file_path: str, key: str, parameter_to_check: str | ||
) -> typing.Any: | ||
reader = MultipleSameKeysIniReader(DUPLICATE_KEYS) | ||
file = glob.glob(os.path.join(study_path, file_path))[0] | ||
path = Path(file) | ||
data = reader.read(path) | ||
return data[key][parameter_to_check] | ||
|
||
|
||
sep = os.sep | ||
other_preferencies = "other preferences" | ||
general_data_path = f"settings{sep}generaldata.ini" | ||
adequacy_patch = "adequacy patch" | ||
mapping_transmission_capacities = { | ||
True: "local-values", | ||
False: "null-for-all-links", | ||
"infinite": "infinite-for-all-links", | ||
} | ||
|
||
|
||
def upgrade_700(study_path: str) -> None: | ||
# It's the basecase study so we pass | ||
pass | ||
|
||
|
||
def upgrade_710(study_path: str) -> None: | ||
geographical_trimming = find_value_in_file( | ||
study_path, general_data_path, "general", "filtering" | ||
) | ||
modify_file( | ||
study_path, | ||
general_data_path, | ||
"optimization", | ||
"link-type", | ||
"local", | ||
None, | ||
) | ||
modify_file( | ||
study_path, | ||
general_data_path, | ||
"general", | ||
"geographic-trimming", | ||
geographical_trimming, | ||
"filtering", | ||
) | ||
modify_file( | ||
study_path, | ||
general_data_path, | ||
"general", | ||
"thematic-trimming", | ||
False, | ||
None, | ||
) | ||
modify_file( | ||
study_path, | ||
general_data_path, | ||
other_preferencies, | ||
"hydro-pricing-mode", | ||
"fast", | ||
None, | ||
) | ||
|
||
|
||
def upgrade_720(study_path: str) -> None: | ||
# There is no input modification between the 7.1.0 and the 7.2.0 version | ||
pass | ||
|
||
|
||
def upgrade_800(study_path: str) -> None: | ||
custom_ts_numbers_value = find_value_in_file( | ||
study_path, general_data_path, "general", "custom-ts-numbers" | ||
) | ||
modify_file( | ||
study_path, | ||
general_data_path, | ||
other_preferencies, | ||
"hydro-heuristic-policy", | ||
"accommodate rule curves", | ||
None, | ||
) | ||
modify_file( | ||
study_path, | ||
general_data_path, | ||
"optimization", | ||
"include-exportstructure", | ||
False, | ||
None, | ||
) | ||
modify_file( | ||
study_path, | ||
general_data_path, | ||
"optimization", | ||
"include-unfeasible-problem-behavior", | ||
"error-verbose", | ||
None, | ||
) | ||
modify_file( | ||
study_path, | ||
general_data_path, | ||
"general", | ||
"custom-scenario", | ||
custom_ts_numbers_value, | ||
"custom-ts-numbers", | ||
) | ||
|
||
|
||
def upgrade_810(study_path: str) -> None: | ||
modify_file( | ||
study_path, | ||
general_data_path, | ||
other_preferencies, | ||
"renewable-generation-modelling", | ||
"aggregated", | ||
None, | ||
) | ||
os.mkdir(os.path.join(study_path + f"{sep}input", "renewables")) | ||
os.mkdir( | ||
os.path.join(study_path + f"{sep}input{sep}renewables", "clusters") | ||
) | ||
os.mkdir(os.path.join(study_path + f"{sep}input{sep}renewables", "series")) | ||
# TODO Cannot update study with renewables clusters for the moment | ||
|
||
|
||
def upgrade_820(study_path: str) -> None: | ||
links = glob.glob(os.path.join(study_path, f"input{sep}links{sep}*")) | ||
if len(links) > 0: | ||
for folder in links: | ||
all_txt = glob.glob(os.path.join(folder, "*.txt")) | ||
if len(all_txt) > 0: | ||
os.mkdir(os.path.join(folder, "capacities")) | ||
for txt in all_txt: | ||
df = pandas.read_csv(txt, sep="\t", header=None) | ||
df_parameters = df.iloc[:, 2:8] | ||
df_direct = df.iloc[:, 0] | ||
df_indirect = df.iloc[:, 1] | ||
reversed_txt = txt[::-1] | ||
k = 0 | ||
while reversed_txt[k] != sep: | ||
k += 1 | ||
name = reversed_txt[4:k][::-1] | ||
numpy.savetxt( | ||
folder + f"{sep}{name}_parameters.txt", | ||
df_parameters.values, | ||
delimiter="\t", | ||
fmt="%.6f", | ||
) | ||
numpy.savetxt( | ||
folder + f"{sep}capacities{sep}{name}_direct.txt", | ||
df_direct.values, | ||
delimiter="\t", | ||
fmt="%.6f", | ||
) | ||
numpy.savetxt( | ||
folder + f"{sep}capacities{sep}{name}_indirect.txt", | ||
df_indirect.values, | ||
delimiter="\t", | ||
fmt="%.6f", | ||
) | ||
os.remove(folder + f"{sep}{name}.txt") | ||
|
||
|
||
def upgrade_830(study_path: str) -> None: | ||
modify_file( | ||
study_path, | ||
general_data_path, | ||
"optimization", | ||
"include-split-exported-mps", | ||
False, | ||
None, | ||
) | ||
modify_file( | ||
study_path, | ||
general_data_path, | ||
adequacy_patch, | ||
"include-adq-patch", | ||
False, | ||
None, | ||
) | ||
modify_file( | ||
study_path, | ||
general_data_path, | ||
adequacy_patch, | ||
"set-to-null-ntc-between-physical-out-for-first-step", | ||
True, | ||
None, | ||
) | ||
modify_file( | ||
study_path, | ||
general_data_path, | ||
adequacy_patch, | ||
"set-to-null-ntc-from-physical-out-to-physical-in-for-first-step", | ||
True, | ||
None, | ||
) | ||
areas = glob.glob(os.path.join(study_path, f"input{sep}areas{sep}*")) | ||
if len(areas) > 0: | ||
for folder in areas: | ||
if Path(folder).is_dir(): | ||
writer = IniWriter() | ||
writer.write( | ||
{"adequacy-patch": {"adequacy-patch-mode": "outside"}}, | ||
Path(folder) / "adequacy_patch.ini", | ||
) | ||
|
||
|
||
def upgrade_840(study_path: str) -> None: | ||
old_value = find_value_in_file( | ||
study_path, | ||
general_data_path, | ||
"optimization", | ||
"transmission-capacities", | ||
) | ||
modify_file( | ||
study_path, | ||
general_data_path, | ||
"optimization", | ||
None, | ||
None, | ||
"include-split-exported-mps", | ||
) | ||
modify_file( | ||
study_path, | ||
general_data_path, | ||
"optimization", | ||
"transmission-capacities", | ||
mapping_transmission_capacities[old_value], | ||
None, | ||
) | ||
|
||
|
||
upgrade_methods = { | ||
700: upgrade_700, | ||
710: upgrade_710, | ||
720: upgrade_720, | ||
800: upgrade_800, | ||
810: upgrade_810, | ||
820: upgrade_820, | ||
830: upgrade_830, | ||
840: upgrade_840, | ||
} | ||
|
||
|
||
class InvalidUpgrade(HTTPException): | ||
def __init__(self, message: str) -> None: | ||
super().__init__(HTTPStatus.UNPROCESSABLE_ENTITY, message) | ||
|
||
|
||
def upgrade_study(study_path: str, new_version: int) -> None: | ||
old_version = get_current_version(study_path) | ||
check_upgrade_is_possible(old_version, new_version) | ||
return do_upgrade(study_path, old_version, new_version) | ||
|
||
|
||
def get_current_version(study_path: str) -> int: | ||
file = glob.glob(os.path.join(study_path, "study.antares")) | ||
if len(file) != 1: | ||
raise StudyValidationError("The path of your study is not valid") | ||
f = open(file[0]) | ||
for line in f: | ||
if "version" in line: | ||
return int(line[10:]) | ||
raise StudyValidationError( | ||
"Your study.antares file is not in the good format" | ||
) | ||
|
||
|
||
def check_upgrade_is_possible(old_version: int, new_version: int) -> None: | ||
if new_version not in upgrade_methods.keys(): | ||
raise InvalidUpgrade(f"The version {new_version} is not supported") | ||
if old_version < 700 or new_version < 700: | ||
raise InvalidUpgrade( | ||
"Sorry the first version we deal with is the 7.0.0" | ||
) | ||
elif old_version > new_version: | ||
raise InvalidUpgrade("Cannot downgrade your study version") | ||
elif old_version == new_version: | ||
raise InvalidUpgrade( | ||
"The version you asked for is the one you currently have" | ||
) | ||
|
||
|
||
def update_study_antares_file(new_version: int, study_path: str) -> None: | ||
epoch_time = datetime(1970, 1, 1) | ||
delta = int((datetime.now() - epoch_time).total_seconds()) | ||
file = glob.glob(os.path.join(study_path, "study.antares"))[0] | ||
with open(file, "r") as f: | ||
lines = f.readlines() | ||
lines[1] = f"version = {new_version}\n" | ||
lines[4] = f"lastsave = {delta}\n" | ||
with open(file, "w") as f: | ||
for item in lines: | ||
f.write(item) | ||
f.close() | ||
|
||
|
||
def do_upgrade(study_path: str, old_version: int, new_version: int) -> None: | ||
update_study_antares_file(new_version, study_path) | ||
possibilities = list(upgrade_methods.keys()) | ||
start = 0 | ||
end = len(possibilities) - 1 | ||
while possibilities[start] != old_version: | ||
start += 1 | ||
while possibilities[end] != new_version: | ||
end -= 1 | ||
return recursive_changes(possibilities[start + 1 : end + 1], study_path) | ||
|
||
|
||
def recursive_changes(update_list: typing.List[int], study_path: str) -> None: | ||
if len(update_list) > 0: | ||
elt = update_list[0] | ||
upgrade_methods[elt](study_path) | ||
recursive_changes(update_list[1:], study_path) | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.