Add tolerance-driven bilinear approximation config API#122
Conversation
Adds the bilinear (flow*head) hydro dispatch formulation and folds in adjacent refactors merged through this branch: - Hydro and storage updates to IOM helpers (#97) - POM-to-IOM type-dispatch API migration - MarketBidCost / ImportExportCost static/TS split + IEC refactor - Shiftable-load interval indexing and validation fixes - HDF system serialization (#75) - Pin GitHub revisions; bridge IOM system-query stubs to PSY public API Co-Authored-By: Luke Kiernan <86331877+luke-kiernan@users.noreply.github.com> Co-Authored-By: Rodrigo Henríquez-Auba <rodrigo.henriquezauba@nrel.gov> Co-Authored-By: Jose Daniel Lara <jdlara@berkeley.edu> Co-Authored-By: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
# Conflicts: # src/PowerOperationsModels.jl # src/static_injector_models/hydro_generation.jl # src/static_injector_models/hydrogeneration_constructor.jl
Use POM.SECONDS_IN_HOUR / POM.M3_TO_KM3 in the m^3/s -> km^3 water-balance conversion instead of bare 3600 / 1e-9 literals, and switch the volume-closure check to isapprox(...; atol = tol) with a named tolerance and a brief comment on its sizing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…pproximation attribute Drop "Bin2" from the formulation type name: the bilinear-approximation scheme is now a `DeviceModel` attribute, not part of the type identity. Defaults reproduce the prior IOM.Bin2Config(IOM.SolverSOS2QuadConfig(4)) behavior bit-for-bit: bilinear_approximation = "bin2" bilinear_quadratic_method = "solver_sos2" bilinear_n_segments = 4 bilinear_add_mccormick = nothing # IOM struct default bilinear_epigraph_depth = nothing # IOM struct default Sentinel `nothing` defers to the IOM constructor's default. A small private builder pair (`_build_quadratic_config`, `_build_bilinear_config`) maps the string attribute values to IOM config objects so callers can swap methods without depending on InfrastructureOptimizationModels. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
A couple of things. This branch should merge to main or to the bilinear branch? I went back and forth with Claude for some ideas on what to do here, and this is a good summary. @jd-lara will need to discuss the approach here for configuring the bilinear approx. Here is a summary: Problem statementThe current
Evaluated optionsOption 1 — Named formulation structs (Anthony's Option A, enhanced)# One struct per top-level method
struct HydroTurbineBin2Dispatch <: AbstractHydroDispatchFormulation end
struct HydroTurbineHybSDispatch <: AbstractHydroDispatchFormulation end
struct HydroTurbineNMDTDispatch <: AbstractHydroDispatchFormulation end
struct HydroTurbineDNMDTDispatch <: AbstractHydroDispatchFormulation end
get_default_attributes(::Type{<:PSY.HydroTurbine}, ::Type{HydroTurbineBin2Dispatch}) =
Dict{String,Any}("n_segments" => 4, "quad_method" => "solver_sos2")Pros:
Cons:
Option 2 — POM config structs, single attribute key ✅ RECOMMENDEDDefine a small hierarchy of config structs in abstract type AbstractBilinearApproxConfig end
"""Bin2 bilinear approximation (default). Linearizes f×h using a two-variable PWL scheme."""
Base.@kwdef struct Bin2Config <: AbstractBilinearApproxConfig
n_segments::Int = 4
quad_method::Symbol = :solver_sos2 # :solver_sos2 | :manual_sos2 | :sawtooth | :epigraph
add_mccormick::Bool = true
end
"""Hybrid-S bilinear approximation."""
Base.@kwdef struct HybSConfig <: AbstractBilinearApproxConfig
n_segments::Int = 4
quad_method::Symbol = :solver_sos2
epigraph_depth::Int # required; no default in IOM
add_mccormick::Bool = false
end
"""NMDT bilinear approximation."""
Base.@kwdef struct NMDTConfig <: AbstractBilinearApproxConfig
depth::Int = 4
end
"""DNMDT bilinear approximation."""
Base.@kwdef struct DNMDTConfig <: AbstractBilinearApproxConfig
depth::Int = 4
end
"""Pass the quadratic flow×head term directly (no MILP linearization)."""
struct NoBilinearApprox <: AbstractBilinearApproxConfig end
function get_default_attributes(
::Type{<:PSY.HydroTurbine}, ::Type{HydroTurbineMILPBilinearDispatch})
return Dict{String,Any}("bilinear_config" => Bin2Config())
end
_iom_config(cfg::Bin2Config) =
IOM.Bin2Config(
_iom_quad_config(cfg.quad_method, cfg.n_segments),
cfg.add_mccormick,
)
_iom_config(cfg::NMDTConfig) = IOM.NMDTBilinearConfig(cfg.depth)
_iom_config(cfg::DNMDTConfig) = IOM.DNMDTBilinearConfig(cfg.depth)
_iom_config(::NoBilinearApprox) = IOM.NoBilinearApproxConfig()
# ...User API: # Common case — no attributes needed
set_device_model!(template, HydroTurbine, HydroTurbineMILPBilinearDispatch)
# Custom segment count
set_device_model!(template, HydroTurbine, HydroTurbineMILPBilinearDispatch;
attributes = Dict("bilinear_config" => POM.Bin2Config(n_segments = 8)))
# Switch to NMDT with depth 6
set_device_model!(template, HydroTurbine, HydroTurbineMILPBilinearDispatch;
attributes = Dict("bilinear_config" => POM.NMDTConfig(depth = 6)))
# HybS — epigraph_depth is required
set_device_model!(template, HydroTurbine, HydroTurbineMILPBilinearDispatch;
attributes = Dict("bilinear_config" => POM.HybSConfig(epigraph_depth = 3)))Pros:
Cons:
Option 3 — Named formulation types aliased to config-struct dispatchCombine both: keep Not recommended — the config struct alone already covers the discoverability goal. RecommendationImplement Option 2 (POM config structs) in a follow-up to PR 122 (or fold it into this PR |
There was a problem hiding this comment.
Pull request overview
Adds a new hydro turbine formulation that routes turbine-flow × reservoir-head bilinear products through configurable IOM approximation schemes, plus tests around the new formulation and hydro unit conversions.
Changes:
- Introduces and exports
HydroTurbineMILPBilinearDispatch. - Adds default attributes and config-building logic for selectable bilinear/quadratic approximation methods.
- Adds hydro tests for MILP bilinear dispatch plumbing and solving.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
src/core/formulations.jl |
Defines the new MILP bilinear hydro turbine formulation and adds it to shared water formulations. |
src/PowerOperationsModels.jl |
Exports the new formulation. |
src/static_injector_models/hydrogeneration_constructor.jl |
Includes the new formulation in turbine constructor plumbing. |
src/static_injector_models/hydro_generation.jl |
Adds approximation attributes, config builders, and the MILP bilinear turbine-power constraint. |
test/test_device_hydro_constructors.jl |
Adds/updates hydro tests for bounds, unit conversion, and MILP bilinear solving. |
test/runtests.jl |
Updates commented disabled-test list entries. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Replace bilinear_n_segments attribute (default 4) with bilinear_tolerance (default 1e-2) on HydroTurbineMILPBilinearDispatch. Users pick the desired approximation gap; the constraint constructor combines that with the per-device flow and head bounds to size each method's discretization automatically. - _build_quadratic_config(method, n, epi) becomes _build_quadratic_config(method, tolerance, max_delta, epi); switches to IOM's kwargs tolerance constructors. - _build_bilinear_config(model) becomes _build_bilinear_config(model, max_delta_x, max_delta_y). For Bin2/HybS the inner quad uses max_delta_x + max_delta_y (widest of x, y, x+/-y) so the requested tolerance holds for all three terms. - In add_constraints! the per-device flow_delta and head_delta (worst case across the turbine's reservoirs) are computed before the config is built. - Repoint IOM source in Project.toml to the ac/tolerance-option branch on Sienna-Platform, which carries the new kwargs API this PR depends on. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
@acostarelli Can you fix the conflicts here? Also, can we have a docstring or some documentation on a rough idea on how tolerance translates to depth or segments |
|
@rodrigomha Yes, I'm still adjusting the IOM side. |
Select the HydroTurbineMILPBilinearDispatch bilinear approximation through a single typed "bilinear_config" attribute (Bin2Config/HybSConfig/NMDTConfig/ DNMDTConfig/NoBilinearApprox) instead of four string attributes, addressing Rodrigo's review on #122. The inner quadratic method is also a POM marker type with per-scheme validity enforced at construction via Union-typed fields. _iom_config dispatches on the config type to build the IOM config, replacing the string-branching _build_bilinear_config. Also validate finite flow/head bounds before sizing the approximation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
# Conflicts: # Project.toml # src/PowerOperationsModels.jl # src/core/formulations.jl # src/static_injector_models/hydro_generation.jl # src/static_injector_models/hydrogeneration_constructor.jl # test/test_device_hydro_constructors.jl
Drop _build_inner_quad: the updated ac/tolerance-option branch sizes
tolerance_depth(Bin2Config{Q}) for NMDT/DNMDT inner quads as two-sided,
so they no longer need epigraph_depth=0 forcing and just build as Q(; depth).
Drop the add_mccormick field/plumbing for now (defer to IOM defaults; TODO
to decide enablement via the tolerance helper). Rename the _iom_config
positional deltas to generic delta_x/delta_y. Remove the three bilinear
config testsets added on this branch.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
| @@ -0,0 +1,237 @@ | |||
| # Bilinear-approximation configuration for `HydroTurbineMILPBilinearDispatch`. | |||
There was a problem hiding this comment.
It is not just for hydro stuff. Make sure none of this is hydro-specific. Yes that's the name of the branch, but it's not the only use-case for this stuff.
| # Bilinear-approximation configuration for `HydroTurbineMILPBilinearDispatch`. | ||
| # | ||
| # These POM-owned types let a user select the bilinear approximation scheme (and | ||
| # its inner quadratic method) for the turbined-flow × head product *by type*, |
| # through the single `"bilinear_config"` `DeviceModel` attribute — without | ||
| # depending on `InfrastructureOptimizationModels` (IOM). The accuracy of each | ||
| # scheme is driven by a `tolerance`; the discretization depth is derived per | ||
| # device at constraint-build time from the tolerance and the device's flow / head |
| """ | ||
| Abstract supertype for the inner quadratic-approximation method used by the | ||
| [`Bin2Config`](@ref) and [`HybSConfig`](@ref) bilinear schemes (those schemes | ||
| approximate `f × h` via squared terms like `(f+h)²`, which each need a quadratic |
There was a problem hiding this comment.
f and h are short for flow and head which, again, are hydro specific
| const Bin2Quad = Union{SolverSOS2, ManualSOS2, Sawtooth, NMDTQuad, DNMDTQuad} | ||
|
|
||
| """ | ||
| Inner quadratic methods valid for [`HybSConfig`](@ref): only the SOS2 variants |
There was a problem hiding this comment.
Please make sure these unions and their doc-strings are up-to-date with what IOM says.
Decouple bilinear approximation config docs/types from the hydro flow×head use case, add tolerance validation, point IOM source back to main, and error early when a turbine has no connected reservoirs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
It seems that we are going to go with the attributes but using tolerance rather than segments and depth. @copilot Review this again |
Reviewed the current state of the PR. Here's a summary: What's in good shape:
One remaining gap to be aware of: |
|
@acostarelli I think @jd-lara prefers that we don't define new structs and just go back to using attributes as before: return Dict{String, Any}(
# Top-level bilinear approximation scheme.
# Supported: "bin2", "hybs", "nmdt", "dnmdt", "none".
"bilinear_approximation" => "bin2",
# Inner quadratic PWL method (used when bilinear_approximation ∈ {"bin2","hybs"}).
# Supported: "solver_sos2", "manual_sos2", "sawtooth", "epigraph",
# "nmdt", "dnmdt", "none".
"bilinear_quadratic_method" => "solver_sos2",
# Maximum approximation gap. Combined with the per-device max_delta
# to size the discretization automatically.
"bilinear_tolerance" => 1e-2,
# `nothing` ⇒ defer to IOM struct default
# (Bin2Config: true; HybSConfig: false; ignored otherwise).
"bilinear_add_mccormick" => nothing,
# `nothing` ⇒ defer to IOM struct default
# (HybSConfig has no default and must be overridden; NMDT/DNMDT
# quadratic: 3*depth; ignored otherwise).
"bilinear_epigraph_depth" => nothing,
)
end@jd-lara Can you confirm if you prefer going back to this, or do you want to use the proposed approach here of adding structs. |
correct, lets keep it as attributes for the public API |
|
These two aren't needed The epigraph depth should be based on the tolerance too |
|
Performance Results
|
Port QuadraticLossConverterMILP and HVDCTwoTerminalVSCLP from a hardcoded SOS2 depth (DEFAULT_INTERPOLATION_LENGTH) to the same tolerance/attribute API used by HydroTurbineMILPBilinearDispatch: "bilinear_approximation", "bilinear_quadratic_method", "bilinear_tolerance". All five schemes are supported. The squares-based schemes (bin2, hybs, none) reuse the standalone loss i_sq via IOM's precomputed (xsq, ysq) overload; the discretization-based schemes (nmdt, dnmdt) never build i², so they take the raw form with nothing to duplicate. The precomputed-vs-raw branch is centralized in _add_converter_bilinear!, and config construction dispatches on the formulation type so the *NLP types stay exact. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The single absolute bilinear_tolerance was an absolute gap on the v·I / flow·head product, whose magnitude differs by formulation, so the same 1e-2 was far stricter for converters (depth 15, intractable) than for hydro. Replace it across all bilinear formulations (HydroTurbineMILPBilinearDispatch, QuadraticLossConverterMILP, HVDCTwoTerminalVSCLP) with two keys: bilinear_relative_tolerance (default 0.05, a fraction of the product magnitude and the default sizing knob) and bilinear_absolute_tolerance (optional). A relative tolerance is scaled to absolute by the term magnitude via the new _resolve_tolerance / _max_abs helpers (max|x|·max|y| for the bilinear, max|i|² for the standalone I² loss term); when both are set the finer binds. The gap→depth inversion stays in IOM; POM does the relative→absolute scaling since it needs the bounds. Default depth drops from 15 to 5 on the converter test systems. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Collapse each NLP/MILP formulation pair into one formulation whose "bilinear_approximation" attribute defaults to the exact "none" case (IOM's NoApproximation configs), opting into MILP via a linearizing scheme: - HydroTurbineMILPBilinearDispatch -> HydroTurbineBilinearDispatch - QuadraticLossConverterMILP/NLP -> QuadraticLossConverter - HVDCTwoTerminalVSCLP/NLP -> HVDCTwoTerminalVSC The VSC PQ-capability (exact disk vs octagon) and pq-square registration are re-keyed from formulation-type dispatch to dispatch on the IOM bilinear config type, keeping the exact/approximate split branch-free. Old *MILP/*NLP/*LP names are removed (no aliases). Tests updated to select the MILP path via an explicit attribute. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
| @test isapprox(abs_i_vals, abs.(i_vals); atol = 1e-6) | ||
| end | ||
|
|
||
| @testset "Converter loss: attribute → IOM config bridge" begin |
There was a problem hiding this comment.
These "attribute config bridge" tests are pointless remove them all.
|
|
||
| # Active-power-only networks don't carry reactive variables, so there is no | ||
| # apparent-power limit to enforce. | ||
| _add_vsc_pq_capability!( |
There was a problem hiding this comment.
Not sure what _capability means here. Also I'm not a fan of "pq" being used as the main name here. Maybe I'm not familiar with power systems enough, but does "PQ" communicate the constraint that "P^2+Q^2<=X" ?
…rename VSC helpers - Add shared BILINEAR_APPROX_DEFAULT_ATTRIBUTES constant (single source of the MILP approximation defaults + their documentation); merge it into the hydro, QuadraticLossConverter, and VSC get_default_attributes instead of duplicating. - Shorten the three formulation docstrings to reference the constant. - _resolve_tolerance now requires exactly one of absolute/relative (error on both or neither) instead of silently taking the min. - Rename the cryptic VSC pq/_capability helpers to apparent-power-limit names (matching HVDCVSCApparentPowerLimitConstraint); update call sites. - Drop three stale comments. - Trim HVDC tests: remove the pure-construction config-bridge testset (replaced by a focused tolerance check), coarsen the MILP solve models, and cover only representative bilinear schemes (bin2 + nmdt). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
| Default `DeviceModel` attributes shared by every formulation that bridges a | ||
| bilinear/quadratic term to IOM's approximation API (`HydroTurbineBilinearDispatch`, | ||
| `QuadraticLossConverter`, `HVDCTwoTerminalVSC`). This is the single source of both | ||
| the default values and the per-attribute documentation; the formulations splice | ||
| it into their `get_default_attributes` (adding only formulation-specific extras). | ||
|
|
||
| - `"bilinear_approximation"` (default `"none"`): the approximation scheme for the | ||
| bilinear product. `"none"` keeps it exact (an NLP needing a nonlinear solver | ||
| such as Ipopt); `"bin2"`, `"hybs"`, `"nmdt"`, `"dnmdt"` are tolerance-driven | ||
| linearizations (mixed-integer linear). | ||
| - `"bilinear_quadratic_method"` (default `"solver_sos2"`): the inner quadratic PWL | ||
| method used by the `"bin2"` and `"hybs"` schemes. Supported: `"solver_sos2"`, | ||
| `"manual_sos2"`, `"sawtooth"`; `"bin2"` also accepts `"nmdt"` and `"dnmdt"`. |
| flow_lb = get_variable_lower_bound(HydroTurbineFlowRateVariable, d, W) | ||
| flow_ub = get_variable_upper_bound(HydroTurbineFlowRateVariable, d, W) | ||
| isnothing(flow_ub) && error( | ||
| "HydroTurbineBilinearDispatch requires finite turbine outflow " * | ||
| "limits to size the bilinear approximation, but turbine \"$(name)\" " * | ||
| "has no `outflow_limits`. Set finite outflow limits or use a " * | ||
| "different hydro turbine formulation.", | ||
| ) | ||
| flow_delta = flow_ub - flow_lb |
The MT-HVDC "QuadraticLossConverter agreement" test compared only the objective, which is dominated by generation cost while the converters carry ~zero current at the optimum (the default CopperPlatePowerModel collapses the two AC islands the DC link bridges, so flow is never needed). The assertion passed vacuously and the accompanying comment rationalized it incorrectly. - Replace it with a conservativeness bound (milp_obj <= nlp_obj * 1.06): the bin2 McCormick relaxation lower-bounds the NLP, allowing for the 5% MIP gap. Use horizon=3h + mip_rel_gap=0.05 so the SOS2 model solves in ~1s instead of timing out. Add a TODO documenting why forced flow is currently unbuildable (DCPPowerModel + VoltageDispatchHVDCNetworkModel fails: QuadraticLossConverter wires into ActivePowerBalance__DCBus, which only the copperplate path creates). - Strengthen the VSC test (genuine forced flow: the VSC replaces a line) to assert the solutions agree, not just objectives: both models push the VSC past 1.5 pu and aggregate throughput agrees within rtol 0.1. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
| # TODO: enable a real forced-flow agreement test for the InterconnectingConverter | ||
| # once DCPPowerModel + VoltageDispatchHVDCNetworkModel can build (the converter | ||
| # needs to inject into whichever DC-bus balance the AC network actually creates), | ||
| # or once a DC-bus load device exists to source current under copperplate. |
Code reviewFound 1 issue:
PowerOperationsModels.jl/src/static_injector_models/hydro_generation.jl Lines 1824 to 1833 in 43465c2 🤖 Generated with Claude Code - If this code review was useful, please react with 👍. Otherwise, react with 👎. |
…erationsModels.jl into ac/hydro-milp-rename
#122 committed InfrastructureOptimizationModels.jl, InfrastructureSystems.jl, and PowerSystems.jl as gitlinks (mode 160000) with no .gitmodules mapping. That broke `Pkg.add`/instantiate of POM main with "cannot get submodules without a working tree". Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Bilinear hydro formulation and IOM/PSY API updates Adds the bilinear (flow*head) hydro dispatch formulation and folds in adjacent refactors merged through this branch: - Hydro and storage updates to IOM helpers (#97) - POM-to-IOM type-dispatch API migration - MarketBidCost / ImportExportCost static/TS split + IEC refactor - Shiftable-load interval indexing and validation fixes - HDF system serialization (#75) - Pin GitHub revisions; bridge IOM system-query stubs to PSY public API Co-Authored-By: Luke Kiernan <86331877+luke-kiernan@users.noreply.github.com> Co-Authored-By: Rodrigo Henríquez-Auba <rodrigo.henriquezauba@nrel.gov> Co-Authored-By: Jose Daniel Lara <jdlara@berkeley.edu> Co-Authored-By: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> * claude initial * fix time-series parameter accessing; fix onparameter cost adding * simplify constructor * formatting * address reviews: add type untyped arguments; support reservation, regularization attributes; enable formatting; use jump utils; port docstrings; * format * refactor: unify two-sided hybrid+storage methods via trait dispatch Collapse paired Charge/Discharge, In/Out, Up/Down add_constraints! and add_to_expression! methods into single bodies that dispatch on small type-keyed trait stubs. Same JuMP shapes, same constraint meta strings, same dispatch reachability. No public API change. Hybrid (src/hybrid_system_models/hybrid_systems.jl): - Charge/DischargeRegularizationConstraint - HybridStorageStatus{Charge,Discharge}OnConstraint - HybridStorage{Charging,Discharging}ReservePowerLimitConstraint (folds in the _ch/_ds_reserve_up_dn_exprs helpers) - HybridTotalReserve{Up,Down}Expression / HybridServedReserve{Out,In}{Up,Down}Expression add_to_expression! - ReserveAssignment/Deployment{Up,Down}{Charge,Discharge} ← Hybrid{Charging,Discharging}ReserveVariable add_to_expression! - HybridStatus{Out,In}OnConstraint Storage (src/energy_storage_models/storage_models.jl): - StorageRegularizationConstraint{Charge,Discharge} Verified: 71/71 hybrid + storage tests pass. Net -376 lines. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * refactor: defer hybrid thermal range constraints to IOM helper Replace three custom hybrid constraint types and their hand-written add_constraints! bodies with a single call to IOM's `add_semicontinuous_range_constraints!`, paralleling how `AbstractThermalUnitCommitment` handles the same range-with-on-variable pattern at thermal_generation.jl:405-419. Mechanism: - Define `get_min_max_limits(::PSY.HybridSystem, ::ActivePowerVariableLimitsConstraint, ::AbstractHybridFormulation)` to read `PSY.get_active_power_limits(PSY.get_thermal_unit(d))`. IOM's helper picks up `OnVariable` keyed by `PSY.HybridSystem` automatically. - For the with-reserves case, introduce two expression types subtyping `RangeConstraint{UB,LB}Expressions`: `HybridThermalActivePowerWithReserve{UB,LB}`. Argument-stage `add_expressions!` aggregates `p_th + Σ r_up` (UB) and `p_th − Σ r_dn` (LB) into them, after which IOM's expression-typed dispatch emits `min·on ≤ p_th − r_dn` and `p_th + r_up ≤ max·on` directly. Removes: - HybridThermalOnVariableUbConstraint, HybridThermalOnVariableLbConstraint, HybridThermalReserveLimitConstraint (constraint types + exports) - _thermal_reserve_up_expr / _thermal_reserve_down_expr helpers - Three add_constraints! bodies (~190 lines) Renewable cases stay hand-written for now: IOM's parameterized helper filters by `IS.has_time_series(d, ts_type, ts_name)`, and PSY's HybridSystem doesn't expose its inner RenewableDispatch's time series through that accessor. A separate change would be required. Verified: 50/50 hybrid tests pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Revert "refactor: defer hybrid thermal range constraints to IOM helper" This reverts commit 59c994b. * docs: rename Bounds → Domain in HybridDispatchWithReserves docstring Per PR review (acostarelli): "domain" describes the value space of each variable (a discrete set or continuous interval) more accurately than "bounds", which suggests inequality-only. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * refactor: replace isa-on-service with multiple-dispatch helpers Address PR review feedback: - "Don't use `isa`. Add a method to handle this, or restructure existing dispatch." Eliminates every `isa(service, PSY.Reserve{...})` and `service isa skip_kind` site in hybrid_systems.jl. The `_excluded_reserve_kind` trait stub goes away — its information is now encoded by union types in helper method signatures. Five sites refactored, all sharing the same shape (per-direction no-op methods + a fallback `::PSY.Service` work method): - `add_to_expression!` for HybridTotalReserveExpression / HybridServedReserveExpression → `_accumulate_reserve!` - `add_to_expression!` for the eight Reserve*Balance{Up,Down} {Charge,Discharge} expressions → `_balance_term!` plus a `_deployment_factor` per-T trait that replaces the `is_up`/`is_deployment` Booleans - `_renewable_reserve_up/down_expr` → `_renewable_reserve_*_term!` thunks sharing `_accumulate_renewable_reserve!` - `_thermal_reserve_up/down_expr` → analogous restructure - `add_constraints!` for ReserveCoverageConstraint{,EndOfPeriod} → `_init_coverage_container!` + `_emit_coverage_constraint!`; the `(service isa PSY.Reserve) || continue` guard becomes the `::PSY.Service` fallback no-op Helper arguments use concrete types (OptimizationContainer, String, Int, Float64, PSY.Storage, …) plus parametric `::Type{T}`/`::Type{U}`/ `d::V`/`::W` to reduce precompilation overhead. - "Combine these if statements." Merges the two adjacent setup ternaries (`r_ub, r_lb = if has_reserves …` and `con_lb = if has_reserves …`) in `add_constraints!(::Type{HybridStatus{Out,In}OnConstraint}, …)` into a single 3-tuple ternary. Test.detect_ambiguities returns 0; full suite passes (50/50). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * restore tests * refactor: parametric abstract types collapse paired hybrid reserve families Hybrid reserve variables, expressions, and constraints had ~16 paired Charge/Discharge, Up/Down, Total/Served (a.k.a. Assignment/Deployment), and UB/LB singleton structs plus ~14 paired trait helpers that all differed only by which sibling they referenced. Introduce marker singletons for ReserveSide, ReserveDirection, ReserveScale (UnscaledReserve / DeployedReserve), reuse IOM's BoundDirection (UpperBound/LowerBound), and reparametrize the family roots: - ReserveAggregationExpression{D,S,Sd} umbrella with two concrete struct families (HybridPCCReserveExpression, StorageReserveBalanceExpression) covering all 16 historical reserve-expression singletons. - HybridPCCReserveVariable{Sd}, HybridStorageSubcomponentReserveVariable{Sd}, HybridStorageSubcomponentPower{Sd}, RegularizationVariable{Sd}. - HybridStatusOnConstraint{Sd}, HybridStorageStatusOnConstraint{Sd}, HybridStorageReservePowerLimitConstraint{Sd}, RegularizationConstraint{Sd}, HybridThermalOnVariableConstraint{B}. All 34 historical concrete names are retained as const aliases so external imports, `get_expression`/`get_variable`/`get_constraint` lookups, and module exports are byte-compatible. Inside hybrid_systems.jl this lets: - _accumulate_reserve! + _balance_term! collapse into one _add_reserve_term! family (PCC boundary and storage subcomponent share the no-op skip and scale dispatch); - thermal/renewable subcomponent accumulators (10 helpers) collapse into one _subcomponent_reserve_term! / _subcomponent_reserve_expr family parametric on the variable type; - the UB/LB thermal-on-variable add_constraints! methods merge; - ~14 paired trait helpers (storage / PCC / regularization) become parametric single-method definitions; - 5 file-local Union consts (_BalanceUpExpr, _BalanceDownExpr, _BalanceDeploymentExpr, _HybridReserveUpExpr, _HybridReserveDownExpr) and _StorageCharge/DischargeSide Union consts get deleted. Additional cleanups: - get_variable_multiplier hybrid signatures take ::Type{<:Formulation} (matches the rest of POM); all W() instance call-sites become type-keyed. - Three `if W <: ...` body-level subtype checks split into separate parametric dispatched methods (HybridStorageBalanceConstraint, RegularizationConstraint, HybridStatusOnConstraint). - _init_coverage_container! uses lazy_container_addition! (idempotent). - add_proportional_cost!(OnVariable, hybrids) hoists the variant/invariant function-handle selection out of the per-t loop. Net: -142 lines across 5 files. Full Pkg.test passes (13125 / 0 fail / 0 error / 1 pre-existing broken). Zero method ambiguities. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * formatting * rename abstract types * formatting * Address review comments on bilinear hydro test Use POM.SECONDS_IN_HOUR / POM.M3_TO_KM3 in the m^3/s -> km^3 water-balance conversion instead of bare 3600 / 1e-9 literals, and switch the volume-closure check to isapprox(...; atol = tol) with a named tolerance and a brief comment on its sizing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Rename HydroTurbineBin2BilinearDispatch to MILP and expose bilinear_approximation attribute Drop "Bin2" from the formulation type name: the bilinear-approximation scheme is now a `DeviceModel` attribute, not part of the type identity. Defaults reproduce the prior IOM.Bin2Config(IOM.SolverSOS2QuadConfig(4)) behavior bit-for-bit: bilinear_approximation = "bin2" bilinear_quadratic_method = "solver_sos2" bilinear_n_segments = 4 bilinear_add_mccormick = nothing # IOM struct default bilinear_epigraph_depth = nothing # IOM struct default Sentinel `nothing` defers to the IOM constructor's default. A small private builder pair (`_build_quadratic_config`, `_build_bilinear_config`) maps the string attribute values to IOM config objects so callers can swap methods without depending on InfrastructureOptimizationModels. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * switch HydroTurbineMILPBilinearDispatch from n_segments to tolerance - Replace bilinear_n_segments attribute (default 4) with bilinear_tolerance (default 1e-2) on HydroTurbineMILPBilinearDispatch. Users pick the desired approximation gap; the constraint constructor combines that with the per-device flow and head bounds to size each method's discretization automatically. - _build_quadratic_config(method, n, epi) becomes _build_quadratic_config(method, tolerance, max_delta, epi); switches to IOM's kwargs tolerance constructors. - _build_bilinear_config(model) becomes _build_bilinear_config(model, max_delta_x, max_delta_y). For Bin2/HybS the inner quad uses max_delta_x + max_delta_y (widest of x, y, x+/-y) so the requested tolerance holds for all three terms. - In add_constraints! the per-device flow_delta and head_delta (worst case across the turbine's reservoirs) are computed before the config is built. - Repoint IOM source in Project.toml to the ac/tolerance-option branch on Sienna-Platform, which carries the new kwargs API this PR depends on. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Replace bilinear attribute config with POM config structs Select the HydroTurbineMILPBilinearDispatch bilinear approximation through a single typed "bilinear_config" attribute (Bin2Config/HybSConfig/NMDTConfig/ DNMDTConfig/NoBilinearApprox) instead of four string attributes, addressing Rodrigo's review on #122. The inner quadratic method is also a POM marker type with per-scheme validity enforced at construction via Union-typed fields. _iom_config dispatches on the config type to build the IOM config, replacing the string-branching _build_bilinear_config. Also validate finite flow/head bounds before sizing the approximation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Simplify bilinear _iom_config bridge against updated IOM contract Drop _build_inner_quad: the updated ac/tolerance-option branch sizes tolerance_depth(Bin2Config{Q}) for NMDT/DNMDT inner quads as two-sided, so they no longer need epigraph_depth=0 forcing and just build as Q(; depth). Drop the add_mccormick field/plumbing for now (defer to IOM defaults; TODO to decide enablement via the tolerance helper). Rename the _iom_config positional deltas to generic delta_x/delta_y. Remove the three bilinear config testsets added on this branch. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Generalize bilinear configs to x×y and validate tolerance Decouple bilinear approximation config docs/types from the hydro flow×head use case, add tolerance validation, point IOM source back to main, and error early when a turbine has no connected reservoirs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor: remove const aliases, use parametric types directly Replace all const aliases for parametric expression and constraint types with their parametric struct definitions. This aligns with project conventions to use parametric types directly rather than const aliases. - Remove ConstraintBound alias (use IOM.BoundDirection directly) - Remove HybridTotalReserveOutUpExpression and similar aliases (use HybridPCCReserveExpression{D, S, Sd} directly) - Remove ReserveAssignmentBalanceUpDischarge and similar aliases (use StorageReserveBalanceExpression{D, S, Sd} directly) - Update all usages in storage, hybrid, and expression handling code - Update documentation references to use base parametric types Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com> * reuse PSY.reservedirection * Return to attribute-based bilinear config, deriving epigraph depth from tolerance Replace the typed "bilinear_config" DeviceModel attribute (POM config structs Bin2Config/HybSConfig/NMDTConfig/DNMDTConfig/NoBilinearApprox and the quad marker types) with the original string-attribute description on HydroTurbineMILPBilinearDispatch: - "bilinear_approximation" ("bin2" | "hybs" | "nmdt" | "dnmdt" | "none") - "bilinear_quadratic_method" ("solver_sos2" | "manual_sos2" | "sawtooth"; "bin2" also accepts "nmdt" | "dnmdt") - "bilinear_tolerance" (finite, > 0) Unlike the original attribute era there is no bilinear_epigraph_depth attribute: every discretization depth (inner quad, HybS's internal epigraph, NMDT/DNMDT) is derived from the tolerance and the per-device flow/head ranges via IOM's tolerance_depth / tolerance_epigraph_depth helpers. bilinear_add_mccormick stays dropped (defers to IOM defaults, TODO retained). core/bilinear_configs.jl is now the string -> IOM-config bridge (_build_bilinear_config), with per-scheme quad validation and tolerance validation at constraint-build time. Adds a testset covering the bridge's happy paths and error paths. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * update source branches * removed excessive comments and tests * remove storage_of helper * Make converter loss approximations attribute-driven Port QuadraticLossConverterMILP and HVDCTwoTerminalVSCLP from a hardcoded SOS2 depth (DEFAULT_INTERPOLATION_LENGTH) to the same tolerance/attribute API used by HydroTurbineMILPBilinearDispatch: "bilinear_approximation", "bilinear_quadratic_method", "bilinear_tolerance". All five schemes are supported. The squares-based schemes (bin2, hybs, none) reuse the standalone loss i_sq via IOM's precomputed (xsq, ysq) overload; the discretization-based schemes (nmdt, dnmdt) never build i², so they take the raw form with nothing to duplicate. The precomputed-vs-raw branch is centralized in _add_converter_bilinear!, and config construction dispatches on the formulation type so the *NLP types stay exact. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Add relative tolerance to bilinear approximation API The single absolute bilinear_tolerance was an absolute gap on the v·I / flow·head product, whose magnitude differs by formulation, so the same 1e-2 was far stricter for converters (depth 15, intractable) than for hydro. Replace it across all bilinear formulations (HydroTurbineMILPBilinearDispatch, QuadraticLossConverterMILP, HVDCTwoTerminalVSCLP) with two keys: bilinear_relative_tolerance (default 0.05, a fraction of the product magnitude and the default sizing knob) and bilinear_absolute_tolerance (optional). A relative tolerance is scaled to absolute by the term magnitude via the new _resolve_tolerance / _max_abs helpers (max|x|·max|y| for the bilinear, max|i|² for the standalone I² loss term); when both are set the finer binds. The gap→depth inversion stays in IOM; POM does the relative→absolute scaling since it needs the bounds. Default depth drops from 15 to 5 on the converter test systems. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Merge NLP+MILP bilinear formulations into single attribute-driven types Collapse each NLP/MILP formulation pair into one formulation whose "bilinear_approximation" attribute defaults to the exact "none" case (IOM's NoApproximation configs), opting into MILP via a linearizing scheme: - HydroTurbineMILPBilinearDispatch -> HydroTurbineBilinearDispatch - QuadraticLossConverterMILP/NLP -> QuadraticLossConverter - HVDCTwoTerminalVSCLP/NLP -> HVDCTwoTerminalVSC The VSC PQ-capability (exact disk vs octagon) and pq-square registration are re-keyed from formulation-type dispatch to dispatch on the IOM bilinear config type, keeping the exact/approximate split branch-free. Old *MILP/*NLP/*LP names are removed (no aliases). Tests updated to select the MILP path via an explicit attribute. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Address June 8 review: centralize bilinear attrs, tighten tolerance, rename VSC helpers - Add shared BILINEAR_APPROX_DEFAULT_ATTRIBUTES constant (single source of the MILP approximation defaults + their documentation); merge it into the hydro, QuadraticLossConverter, and VSC get_default_attributes instead of duplicating. - Shorten the three formulation docstrings to reference the constant. - _resolve_tolerance now requires exactly one of absolute/relative (error on both or neither) instead of silently taking the min. - Rename the cryptic VSC pq/_capability helpers to apparent-power-limit names (matching HVDCVSCApparentPowerLimitConstraint); update call sites. - Drop three stale comments. - Trim HVDC tests: remove the pure-construction config-bridge testset (replaced by a focused tolerance check), coarsen the MILP solve models, and cover only representative bilinear schemes (bin2 + nmdt). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * two-layer -> one-layer helper with extra noop for ambiguity * remove excessive comments and tests * copilot review * fix target for approximation test * Fix vacuous HVDC MILP/NLP agreement test The MT-HVDC "QuadraticLossConverter agreement" test compared only the objective, which is dominated by generation cost while the converters carry ~zero current at the optimum (the default CopperPlatePowerModel collapses the two AC islands the DC link bridges, so flow is never needed). The assertion passed vacuously and the accompanying comment rationalized it incorrectly. - Replace it with a conservativeness bound (milp_obj <= nlp_obj * 1.06): the bin2 McCormick relaxation lower-bounds the NLP, allowing for the 5% MIP gap. Use horizon=3h + mip_rel_gap=0.05 so the SOS2 model solves in ~1s instead of timing out. Add a TODO documenting why forced flow is currently unbuildable (DCPPowerModel + VoltageDispatchHVDCNetworkModel fails: QuadraticLossConverter wires into ActivePowerBalance__DCBus, which only the copperplate path creates). - Strengthen the VSC test (genuine forced flow: the VSC replaces a line) to assert the solutions agree, not just objectives: both models push the VSC past 1.5 pu and aggregate throughput agrees within rtol 0.1. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * remove bad test * fix a few more reserve traits * formatting * tests run faster; use units in getters * Address PR #104 June-9 review: storage reserve bug, storage-less hybrids, SOC target - Fix undefined `Up`/`Down` type params in storage reserve expressions (StorageReserveBalanceExpression{Up/Down,...} -> {PSY.ReserveUp/ReserveDown,...}), which previously errored when constructing storage ancillary services. - C4: create TotalReserveOffering containers for every hybrid that participates in a reserve service, not just hybrids with storage. get_expression_type_for_reserve routes all hybrids' ActivePowerReserveVariable into TotalReserveOffering, so storage-less hybrids with reserves no longer hit a missing-container error. The subcomponent feed stays gated on storage. Adds a regression test. - C7: give the hybrid end-of-period energy target its own HybridEnergyTargetConstraint (a one-sided floor e_T >= E_T, no slacks) instead of reusing the storage StateofChargeTargetConstraint (an equality with surplus/shortfall slacks), and fix the hybrid formulation docstring to match. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Fix hybrid energy target: restore surplus/shortage slacks (soft equality) The HybridDispatchWithReserves port of the end-of-period storage energy target was mis-ported from HybridSystemsSimulations.jl: it dropped the surplus/shortage slack variables and implemented a hard one-sided floor (e_T >= E_T) instead of HSS's soft equality with penalized slacks. The energy_target path was never exercised by any test, so this went unnoticed (and the slack-typed add_variables!/add_constraints! signatures only accepted FlattenIteratorWrapper, never the Vector the constructor passes, so the constraint method never even matched). Mirror POM's storage StateofChargeTargetConstraint, adapted for the hybrid: - Add HybridEnergySurplusVariable / HybridEnergyShortageVariable (non-negative, final-time-step only) and export them. - Make HybridEnergyTargetConstraint a soft equality e_T - e+ + e- = E_T. - Penalize both slacks in the objective from the storage subcomponent's StorageCost (energy_surplus_cost / energy_shortage_cost), gated on energy_target. - Add the slacks in the constructor ArgumentConstructStage. - Broaden the slack add_variables! and the target add_constraints! to accept Vector as well as FlattenIteratorWrapper. Keep the existing target RHS scaling (storage_target is a ratio of capacity in PSY; the hybrid EnergyVariable is absolute energy), which is the one intentional divergence from HSS's raw get_storage_target. Add tests modeled on the storage energy-target tests: assert the slacks exist and the constraint is an equality (would have caught the regression), that the slacks are absent when energy_target=false, and an on-vs-off objective check confirming the penalty reaches the objective. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * formatting * Apply suggestions from code review Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Anthony Costarelli <acostare@nrel.gov> Co-authored-by: Luke Kiernan <86331877+luke-kiernan@users.noreply.github.com> Co-authored-by: Rodrigo Henríquez-Auba <rodrigo.henriquezauba@nrel.gov> Co-authored-by: Jose Daniel Lara <jdlara@berkeley.edu> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Introduces a type-safe, tolerance-driven configuration API for selecting the bilinear approximation method used by
HydroTurbineMILPBilinearDispatch, replacing the previous multi-keyDict{String,Any}attribute surface with a single"bilinear_config"attribute holding a typed config struct.Changes Made
src/core/bilinear_configs.jl: Defines a genericAbstractBilinearApproxConfighierarchy (not hydro-specific) with config structs for each supported scheme:Bin2Config— default; Bin2 bilinear approximation with selectable inner quadratic methodHybSConfig— Hybrid-S approximationNMDTConfig— NMDT approximationDNMDTConfig— DNMDT approximationNoBilinearApprox— passes the product directly to the solver (no MILP linearization)SolverSOS2,ManualSOS2,Sawtooth,Epigraph,NMDTQuad,DNMDTQuad, withBin2Quad/HybSQuadunion type constraints matching IOM's supported combinationstolerance::Float64(default1e-2); the discretization depth is derived automatically from the tolerance and the two variables' domain widths viaIOM.tolerance_depthat constraint-build time — no manual depth or segment-count knob exposed to the user_validate_tolerance: Construction-time guard rejecting non-finite or non-positive tolerances with a clearArgumentError_iom_configbridge: Dispatches onAbstractBilinearApproxConfigsubtypes to produce the corresponding IOM config, replacing prior string-branching logicHydroTurbineMILPBilinearDispatch: Default attribute isDict("bilinear_config" => Bin2Config()); finite flow and head bounds are validated before calling IOM, with clear error messagesProject.toml:InfrastructureOptimizationModelspinned tomainUser API
Testing
HydroTurbineMILPBilinearDispatchplumbing (variable/constraint presence)