Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
123 changes: 63 additions & 60 deletions doc/src/agnostic.rst
Original file line number Diff line number Diff line change
Expand Up @@ -53,71 +53,72 @@ JSON file format
----------------

The directory named in the ``--mps-files-directory`` needs to have
two files for each scenario: and mps file and a json file. The json
two files for each scenario: a mps file and a json file. The json
file need to have certain literal strings as well as scenario-specific
data. In this specification, scenario specific data is named with underscores.
Note that the total number of tree nodes is given as an integer, but the file
only contains the data for nodes for the single scenario.

.. code-block:: json

{
"scenarioData": {
"name": scenario_name,
"scenProb": scenario_probability
},
"treeData": {
"globalNodeCount": number_of_nodes_in_entire_tree,
"nodes: {
"ROOT": {
"condProb": 1.0,
"nonAnts": [
"first_root_node_nonant_name",
"second_root_node_nonant_name",
...
]
}
"ROOT_i": {
"condProb": conditional_probability_of_second_stage_node_i,
"nonAnts": [
first_nonant_name_at_node,
second_node_nonant_name_at_node,
...
]
.. code-block:: python
Copy link
Contributor Author

@mosc9575 mosc9575 Nov 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used the text highlighting for python, because of the use of variables. This looks quite good and the selection "JSON" for text highlighting was broken. I think this is a good compromise.


{
"scenarioData": {
"name": scenario_name,
"scenProb": scenario_probability,
},
"treeData": {
"globalNodeCount": number_of_nodes_in_entire_tree,
"nodes: {
"ROOT": {
"condProb": 1.0,
"nonAnts": [
"first_root_node_nonant_name",
"second_root_node_nonant_name",
#...
]
}
"ROOT_i": {
"condProb": conditional_probability_of_second_stage_node_i,
"nonAnts": [
first_nonant_name_at_node,
second_node_nonant_name_at_node,
#...
]
}
}
}
}
}

Two-stage JSON example
~~~~~~~~~~~~~~~~~~~~~~

Two-stage problems are simple because there is only one node in the scenario tree and its name
must be ROOT. Here is an example

.. code-block:: json
{
"scenarioData": {
"name": "unknown",
"scenProb": 0.3333333333333333
},
"treeData": {
"globalNodeCount": 1,
"nodes": {
"ROOT": {
"serialNumber": 0,
"condProb": 1.0,
"nonAnts": [
"NumProducedFirstStage(1)",
"NumProducedFirstStage(2)",
"NumProducedFirstStage(3)",
...
"NumUnitsCutFirstStage(10_10)"
]
.. code-block:: python

{
"scenarioData": {
"name": "unknown",
"scenProb": 0.3333333333333333
},
"treeData": {
"globalNodeCount": 1,
"nodes": {
"ROOT": {
"serialNumber": 0,
"condProb": 1.0,
"nonAnts": [
"NumProducedFirstStage(1)",
"NumProducedFirstStage(2)",
"NumProducedFirstStage(3)",
# ...
"NumUnitsCutFirstStage(10_10)",
]
}
}
}
}
}


Naming Conventions
Expand Down Expand Up @@ -150,7 +151,8 @@ Assuming support has been added for the desired AML, the modeler supplies
two files:

- a model file with the model written in the guest AML (AMPL example: ``mpisppy.agnostic.examples.farmer.mod``)
- a thin model wrapper for the model file written in Python (AMPL example: ``mpisppy.agnostic.examples.farmer_ampl_model.py``). This thin python wrapper is model specific.
- a thin model wrapper for the model file written in Python (AMPL example: ``mpisppy.agnostic.examples.farmer_ampl_model.py``).
This thin python wrapper is model specific.

There can be a little confusion if there are error messages because
both files are sometimes refered to as the `model file.`
Expand Down Expand Up @@ -187,11 +189,11 @@ original model when updating the objective function. If this is an issue,
you might want to write a problem-specific module to replace the guest
interface and the model wrapper with a single module. For an example, see
``examples.farmer.agnostic.farmer_xxxx_agnostic``, where xxxx is replaced,
e.g., by ampl.
e.g., by ampl.

Architecture
^^^^^^^^^^^^
The following picture presents the architecture of the files.
The following picture presents the architecture of the files.

.. image:: images/agnostic_architecture.png
:alt: Architecture of the agnostic files
Expand Down Expand Up @@ -221,7 +223,7 @@ The use of scenario bundles can dramatically improve the performance
of scenario decomposition algorithms such as PH and APH. Although mpi-sppy
has facitilites for forming bundles, the mpi-sppy
``agnostic`` package assumes that bundles will be completely handled
by the guest. Bundles will be returned by the scenario creator function
by the guest. Bundles will be returned by the scenario creator function
as if they are a scenario. Although it seems sort of like a trick, it is
really the way bundles are intended to operate so we sometimes refer to
`true` bundles, which are used in non-agnostic way as briefly
Expand Down Expand Up @@ -252,18 +254,19 @@ Some notes
^^^^^^^^^^

- The helper function called ``scenario_names_creator`` needs to be co-opted
to instead create bundle names and the code in the scenario_creator function
then needs to create its own scenario names for bundles. At the time
of this writing this results in a major hack being needed in order to
get bundle information to the names creator in the Pyomo example described
below. You need to supply a function called ``bundle_hack`` in your python model file that
does whatever needs to be done to alert the names creator that there
bundles. The function takes the config object as an argument.
See ``mpisppy.agnostic.farmer4agnostic.py``
to instead create bundle names and the code in the scenario_creator function
then needs to create its own scenario names for bundles. At the time
of this writing this results in a major hack being needed in order to
get bundle information to the names creator in the Pyomo example described
below. You need to supply a function called ``bundle_hack`` in your python model file that
does whatever needs to be done to alert the names creator that there
bundles. The function takes the config object as an argument.
See ``mpisppy.agnostic.farmer4agnostic.py``
- There is a heavy bias toward uniform probabilities in the examples and in
the mpi-sppy utilities. Scenario probabilities are attached to the scenario
as ``_mpisppy_probability`` so if your probabilities are not uniform, you will
need to calculate them for each bundle (your EF maker code can do that for you). Note that even if probabilities are uniform for the scenarios, they won't
need to calculate them for each bundle (your EF maker code can do that for you).
Note that even if probabilities are uniform for the scenarios, they won't
be uniform for the bundles unless you require that the bundle size divides
the number of scenarios.
- There is a similar bias toward two stage problems, which is
Expand Down
6 changes: 2 additions & 4 deletions doc/src/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -67,10 +67,8 @@
templates_path = ['_templates']

# The suffix(es) of source filenames.
# You can specify multiple suffix as a list of string:
#
# source_suffix = ['.rst', '.md']
source_suffix = '.rst'
# You can specify multiple suffix as a dict:
source_suffix = {".rst": "restructuredtext"}

# The root toctree document.
root_doc = 'index'
Expand Down
20 changes: 10 additions & 10 deletions doc/src/generic_cylinders.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ The file name is given without the the ``.py`` extension as the
needed even with the ``--help`` argument, e.g.,

.. code-block:: bash

python ../mpisppy/generic_cylinders.py --module-name farmer/farmer --help

.. Note::
Expand Down Expand Up @@ -69,27 +69,27 @@ The class definition needs to include all helper functions other than
demonstrates how to implement a class in the module (although in this
particular example, there is no advantage to doing that).


custom_writer
-------------

This is an advanced topic.
This is an advanced topic.
Advanced users might want to write their own solution output function. If the
module contains a function called ``custom_writer()``, it will be passed
to the solution writer. Up to four functions can be specified in the module (or the
class if you are using a class):

- ef_root_nonants_solution_writer(file_name, representative_scenario, bundling_indicator)
- ef_tree_solution_writer(directory_name, scenario_name, scenario, bundling_indicator)
- first_stage_solution_writer(file_name, scenario,bundling_indicator)
- tree_solution_writer(directory_name, scenario_name, scenario, bundling_indicator)
- ef_root_nonants_solution_writer(file_name, representative_scenario, bundling_indicator)
- ef_tree_solution_writer(directory_name, scenario_name, scenario, bundling_indicator)
- first_stage_solution_writer(file_name, scenario,bundling_indicator)
- tree_solution_writer(directory_name, scenario_name, scenario, bundling_indicator)

The first two, if present, will be used for the EF if that is select
and the second two for hub and spoke solutions. For further
and the second two for hub and spoke solutions. For further
information, look at the code in ``mpisppy.generic_cylinders.py`` to
see how these are used and in ``mpisppy.utils.sputils`` for example functions
such as ``first_stage_nonant_npy_serializer``. There is a very simple
example function in ``examples.netdes.netdes_with_class.py''.
such as ``first_stage_nonant_npy_serializer``. There is a very simple
example function in ``examples.netdes.netdes_with_class.py``.

.. Warning::
These functions will only be used if cfg.solution_base_name has been given a value by the user.
Expand Down
10 changes: 5 additions & 5 deletions doc/src/grad_rho.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ Grad-rho is also supported in ``generic_cylinders.py``.

There are options in ``cfg`` to control dynamic updates:

* `--dynamic-rho-primal-crit` and `--dynamic-rho-dual-crit` are booleans that trigger dynamic rho
* `--dynamic-rho-primal-thresh` and `--dynamic-rho-dual-thresh` control how sensitive the trigger is.
They have default values, so do not need to be set. See
``dyn_rho_base.py`` to see how the update is done if you don't like the default values.
* `--grad-rho-multiplier` is a cummulative multiplier when rho is set or updated.
* ``--dynamic-rho-primal-crit`` and ``--dynamic-rho-dual-crit`` are booleans that trigger dynamic rho
* ``--dynamic-rho-primal-thresh`` and ``--dynamic-rho-dual-thresh`` control how sensitive the trigger is.
They have default values, so do not need to be set. See
``dyn_rho_base.py`` to see how the update is done if you don't like the default values.
* ``--grad-rho-multiplier`` is a cummulative multiplier when rho is set or updated.
8 changes: 4 additions & 4 deletions doc/src/internals.rst
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ Standard Construction

If you are going to use the ``vardatalist`` you don't really need to know how it is constructed, but
we will illustrate one common way here.
Many modelers/users have a call to ``attach_root_node`` in their scenario creator function (e.g., see the scenario scenario
Many modelers/users have a call to ``attach_root_node`` in their scenario creator function (e.g., see the scenario
creator in the farmer example), which is a utility
for two-stage problems only. Here is the function signature:

Expand All @@ -63,9 +63,9 @@ creates the ``nonant_vardata_list`` with this call

.. code-block:: python

self.nonant_vardata_list = build_vardatalist(self,
scen_model,
self.nonant_list)
import mpisppy.utils.sputils as sputils

self.nonant_vardata_list = sputils.build_vardatalist(scen_model, self.nonant_list)

where ``build_vardatalist`` function converts strings to vardata objects and expands indexed Vars.

Expand Down
6 changes: 3 additions & 3 deletions doc/src/properbundles.rst
Original file line number Diff line number Diff line change
Expand Up @@ -61,11 +61,11 @@ Modules
-------

In addition to command line options specified in ``mpisppy.utils.config.py``
and supported in ``mpisppy.generic_cylinders.py'',
and supported in ``mpisppy.generic_cylinders.py``,
there are two modules that have most of the support for proper bundles:

- ``mpisppy.utils.pickle_bundle.py`` has miscellaneous utilities related to picking and other data processing
- ``mpisppy.utils.proper_bundler.py`` has wrappers for cylinder programs
- ``mpisppy.utils.pickle_bundle.py`` has miscellaneous utilities related to picking and other data processing
- ``mpisppy.utils.proper_bundler.py`` has wrappers for cylinder programs


Multistage
Expand Down
27 changes: 13 additions & 14 deletions doc/src/stoch_admmWrapper.rst
Original file line number Diff line number Diff line change
Expand Up @@ -53,13 +53,13 @@ The driver file requires the `scenario_creator function <scenario_creator>`_ whi
.. py:function:: scenario_creator(admm_stoch_subproblem_scenario_name)
:no-index:

Creates the model, which should include the consensus variables.
However, the `scenario_creator` function shouldn't include the consensus variables in the list of stochastic
variables. (E.g., for a
two-stage problem, they should not be in the list supplied to ``attach_root_node``. The
consensus variables
are supplied to `admm_wrapper` in the driver in a list passed to the ``stoch_admmWrapper`` constructor.)
Therefore, for multi-stage stochastics, only the stochastic tree as it would be represented without the admm decomposition needs to be created by the `scenario_creator` function.
Creates the model, which should include the consensus variables. However, the `scenario_creator`
function shouldn't include the consensus variables in the list of stochastic variables. (E.g.,
for a two-stage problem, they should not be in the list supplied to ``attach_root_node``. The
consensus variables are supplied to `admm_wrapper` in the driver in a list passed to the
``stoch_admmWrapper`` constructor.) Therefore, for multi-stage stochastics, only the stochastic
tree as it would be represented without the admm decomposition needs to be created by the
`scenario_creator` function.

Args:
admm_stoch_subproblem_scenario_name (str): the name of the extended scenario that will be created.
Expand All @@ -74,15 +74,14 @@ Here is a summary of helper functions:
* ``scenario_creator_kwargs`` (dict[str]): key words arguments needed in ``scenario_creator``

* A function that is called at termination in some modules (e.g. PH)
.. py:function:: scenario_denouement
:no-index:

Args:
rank (int): rank in the cylinder

admm_stoch_subproblem_scenario_name (str): name of the extended scenario
.. py:function:: scenario_denouement
:no-index:

scenario (Pyomo ConcreteModel): the instantiated model
Args:
rank (int): rank in the cylinder
admm_stoch_subproblem_scenario_name (str): name of the extended scenario
scenario (Pyomo ConcreteModel): the instantiated model

* ``stoch_scenario_names``, ``admm_subproblem_names`` and ``all_admm_stoch_subproblem_scenario_names`` (lists of str)

Expand Down
Loading
Loading