Improvements to solve_strongly_connected_components

[dallan-keylogic]

Summary

Rationale

While troubleshooting the BlockTriangularizationInitializer in IDAES, I encountered some issues with the default troubleshooting options in solve_strongly_connected_components. I also discovered some shortcomings of the algorithm that should raise warnings or exceptions.

Description

1. There should be a better way see the logged information about the 1x1 and larger blocks that are solved than setting the global logging level to DEBUG. Setting the global logging level also prints information about creating the temporary blocks to be solved and attaching variables/constraints to them which clogs up the log.
2. For the 1x1 blocks, the names of both the variable and constraint should be logged.
3. For the N x N blocks, a preview of the variable and constraint names (like the first 10 in each list) should be logged.
4. calculate_variable_from_constraint does not necessarily respect bounds. Therefore, solve_strongly_connected_components should monitor its output and raise an exception if a "large" bound violation is encountered. (I often see "small" bound violations, like -1e-14 for a variable with a lower bound of 0, because the Newton algorithm does not continue iterating once the residual is sufficiently small. These should be ignored.)
5. A larger change would be adding generate_strongly_connected_components and solve_strongly_connected_components to an object so that the strongly connected components and igraph could be cached. I'd like to run generate_strongly_connected_components first, iterate over the strongly connected components to see the system size, then log a Warning if the block size is over some configured value (default=20?).

To address (2) and (3), I've modified my local copy to contain the following code:

        with TemporarySubsystemManager(to_fix=inputs, remove_bounds_on_fix=True):
            N = len(scc.vars)
            if N == 1 and use_calc_var:
                if log_blocks:
                    _log.debug(f"Solving 1x1 block with {scc.vars[0].name} and {scc.cons[0].name}.")
                results = calculate_variable_from_constraint(
                    scc.vars[0], scc.cons[0], **calc_var_kwds
                )
            else:
                if solver is None:
                    var_names = [var.name for var in scc.vars.values()][:10]
                    con_names = [con.name for con in scc.cons.values()][:10]
                    raise RuntimeError(
                        "An external solver is required if block has strongly\n"
                        "connected components of size greater than one (is not"
                        " a DAG).\nGot an SCC of size %sx%s including"
                        " components:\n%s\n%s" % (N, N, var_names, con_names)
                    )
                if log_blocks:
                    var_names = [var.name for var in scc.vars.values()][:10]
                    con_names = [con.name for con in scc.cons.values()][:10]
                    _log.debug(f"Solving {N}x{N} block including \n{var_names}\n{con_names}.")
                results = solver.solve(scc, **solve_kwds)
            res_list.append(results)

If it is useful, it can be used or altered.

[Robbybp]

I think (4) is really two issues:

1. solve_strongly_connected_components is not a "solver", so it makes no promise about respecting bounds. This is handled entirely by the "subsolver".
2. The algorithm is not robust to the combination of bounds and solution multiplicity. I've opened Handle bounds in solve_strongly_connected_components #3573 to address this.