Broyden updating provides all objective function lowering late in calibration (with PEST_HP)

[sjepsen395]

All,

I've been seeing an interesting pattern lately in our calibration of a large scale integrated hydrologic model using PEST_HP. We have Broyden updating turned on from the start and a switch to 3-point derivatives (this kicks in usually around iteration 8 out of typically ~25). We have not yet tried 5-point derivates. Late in the calibration, 100% of objective function lowering takes place only after the Broyden update of the Jacobian matrix. By this time in the calibration, the rate of objective lowering is low (e.g., 1% per iteration) but still enough to be important for our work.

I feel like this is saying something fundamental about our model or calibration, but I'm not sure what that might be. Maybe something about the shape of the objective function surface (egg crates everywhere) late in the calibration? Could it suggest that higher order derivatives (5-point) may be particularly helpful in cases where Broyden does all the Phi lowering? I'd be interested to hear any insights into this and/or references relating to this pattern of late calibration lowering with only Broyden updating turned on.

Thanks,
Steve

[doherty1234]

Hello
I personally find that there are many cases where Broyden updating does not help much - and some where it helps a lot (in fact it makes updating possible). These tend to be cases where the inverse problem is highly nonlinear. This can occur, for example, where depending on parameter values, a few Jacobian singular values are high and the others are low but where this disposition of singular value "energy" can be very different for other parameter values. So a combination of lots of different parameter upgrade directions (set UPTESTMIN appropriately) and Broyden updating can really make a difference if you are updating from a parameter for which the Jacobian matrix is a little locally insensitivity with respect to some parameters.
Other times nonlinearity alone can be the reason why Broyden updating works so well.
As for "egg crates". These can be real or they may be an outcome of poor finite difference derivatives caused by model numerical idioscyncrasies. In the latter case JACTEST may help to understand the situation.
But, as stated, this seems to happen most in difficult cases.

[sjepsen395]

Hi doherty1234,

Thanks for your reply. We're currently running >300 CPUs on the cloud, letting PEST_HP decide on how many parameter upgrade directions to test. It always chooses a number between 140 and ~150. It seems like a small number of these parameter upgrades test the "magic value" that allows objective lowering--that is, when Broyden is doing all objective lowering late in the calibration. Two things I could try to improve lowering during this part of the calibration are: (1) set UPTESTMIN to 319 for a 320-CPU run to get many more parameter upgrade directions and increased likelihood of hitting a "magic value" (will PEST_HP go through with that request?), and (2) try 5-point derivatives from a restart run with FORCEN = force_5 , just to see if a high order derivative improves lowering late in the calibration when Broyden is providing all lowering. Idea #2 could be tested after the calibration, if resources allow.

We're aware of complications arising from engine automatic adjustments (irrigation scheduling, rapid movement of GW to control heads above ground surface) that introduce a type of nonlinearity into the model response. Unfortunately, there is not much we can do about that as these adjustments must be turned on for models to serve their purpose. I have used JACTEST in the past to look into some of this (looking into numerical granularity, etc. etc.), seeing some spurious model observations for certain parameter values that I attributed to these types of automatic adjustments.

A possibly related control that just came to mind is split slope analysis. We have this turned on. But we don't really know how much it's helping, if it all. But the concept seems good for possibly helping spurious model responses caused by engine automatic adjustments.

Thanks,
Steve

[doherty1234]

Actually, UPTESTMIN is a tricky setting. You are right - the "pellets in the shotgun" the more likely it is that you will stumble across the the best direction in a space where the objective function surface has lots of small corrugations. However, sometimes the model may not like some of these parameter choices and runs can last a long time. Anyway, you seem to be doing the right thing and testing out all possibilities.
Good luck
John

[sjepsen395]

Hi John,

With cloud computing and other resources nowadays, I wonder if it would make sense for PEST_HP to use as many available CPUs as possible for the parameter upgrade calculations. For that, I was envisioning finer differences between Marquardt Lambdas and upgrade fractions being tested. There may well be some numerical complications that I'm not considering here. I have seen a wide range in model run times (~ 100-225% of normal) during parameter upgrades, as you pointed out. Luckily the only model run failures have been those caused by me in letting some parameters take on negative values in PLPROC (my coding laziness caught up with me).

Thanks for all your input,
Steve