Skip to content

Granular (Upper and lower) Sensitivity#35

Draft
jonasViehweger wants to merge 2 commits intoec-jrc:masterfrom
jonasViehweger:feat/granular-sensitivity
Draft

Granular (Upper and lower) Sensitivity#35
jonasViehweger wants to merge 2 commits intoec-jrc:masterfrom
jonasViehweger:feat/granular-sensitivity

Conversation

@jonasViehweger
Copy link
Contributor

@jonasViehweger jonasViehweger commented Feb 12, 2026

For some inputs (TCW, NDMI) and some types of seasonal trajectories, the direction of the residuals which signal a true break and not time-series artifacts like seasonality or missed clouds/snow is uni-directional.

For example with NDMI, true disturbances will almost always signal through sustained lower NDMI values.
Another case is residual distributions. Even though residual distributions will be centered around a mean of 0, the distribution of residuals might be quite lopsided. My guess is that this is one of the reasons why IQR performs so well out of the box; it has some amount of granular upper and lower sensitivity built in through its statistical limits.

I carried out a hyperparameter tuning test for IQR using the jr-digital/DISFOR dataset, which showed that the optimal parameters for this test set using NDMI as input and optimizing for F1 score, results in lopsided sensitivities:

{'sensitivity_lower': 2.0050327696845214,
 'sensitivity_upper': 4.259944838204534,
 'trend': False,
 'harmonic_order': 2,
 'boundary': 7,
 'L': 5.237738192464925}

compared to the optimal parameters with just a single sensitivity value for both positive and negative deviations

{'sensitivity': 2.6408646132552693,
 'trend': False,
 'harmonic_order': 4,
 'boundary': 8,
 'L': 18.57921286921607}

Being able to set sensitivities individually yields an improvement in F1 of around 3% (0.762455 with separate sensitivities vs. 0.731636).

I've implemented this additional behaviour for IQR in this pull request. There's a few things left to clear up before doing this same treatment for every class:

  • Should the lower sensitivity be defined as (-lower, +upper) or how it is right now as (lower, upper) and the directionality is handled in the sensitivity calculation logic in _update_process().
  • Does the docstring describe the behavirous clearly?
  • Right now the Integration test is failing due to the export/import not supporting a tuple for sensitivity. I would fix that once the sensitivity is implemented for all classes.

Any thoughts @loicdtx?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant