Granular (Upper and lower) Sensitivity#35
Draft
jonasViehweger wants to merge 2 commits intoec-jrc:masterfrom
Draft
Granular (Upper and lower) Sensitivity#35jonasViehweger wants to merge 2 commits intoec-jrc:masterfrom
jonasViehweger wants to merge 2 commits intoec-jrc:masterfrom
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
For some inputs (TCW, NDMI) and some types of seasonal trajectories, the direction of the residuals which signal a true break and not time-series artifacts like seasonality or missed clouds/snow is uni-directional.
For example with NDMI, true disturbances will almost always signal through sustained lower NDMI values.
Another case is residual distributions. Even though residual distributions will be centered around a mean of 0, the distribution of residuals might be quite lopsided. My guess is that this is one of the reasons why IQR performs so well out of the box; it has some amount of granular upper and lower sensitivity built in through its statistical limits.
I carried out a hyperparameter tuning test for IQR using the jr-digital/DISFOR dataset, which showed that the optimal parameters for this test set using NDMI as input and optimizing for F1 score, results in lopsided sensitivities:
compared to the optimal parameters with just a single sensitivity value for both positive and negative deviations
Being able to set sensitivities individually yields an improvement in F1 of around 3% (0.762455 with separate sensitivities vs. 0.731636).
I've implemented this additional behaviour for IQR in this pull request. There's a few things left to clear up before doing this same treatment for every class:
_update_process().Any thoughts @loicdtx?