uncertainty/certainty score for annotations

[Elleiv]

Working with low-quality videos, for which annotations are sometimes uncertain (for example: is this the left hind leg or just a shadow?), I was wondering whether it would be possible to add an uncertainty/certainty score to annotations. This would allow to give more weight in model learning to annotations for which we are certain. What do you think? Feasible? Really useful?

Furthermore, as a daily sleap user (thank you very much for your work!) two very small GUI improvements could drastically improve my experience: 1- the ability to modify the preview of a video (eg, contrast, brightness) directly in the GUI (For now, I have to pre-process a large number of videos before using SLEAP just for brightness) and 2- in the 'run traning' box, the ability to customize where and how many predictions to make.

Thanks!

[talmo]

Hi @Elleiv,

Working with low-quality videos, for which annotations are sometimes uncertain (for example: is this the left hind leg or just a shadow?), I was wondering whether it would be possible to add an uncertainty/certainty score to annotations. This would allow to give more weight in model learning to annotations for which we are certain. What do you think? Feasible? Really useful?

That's a great suggestion! We originally added some data structures to support this, but we didn't end up implementing it. Part of what we didn't work through is what the UX would be like -- how do you actually assign confidences in the GUI? Right now we just have right-click to toggle visibility, which is a special case of setting the confidence to 0 or 1. To do intermediate values, it would get tricky (maybe just one mid level 0.5 with a shortcut?) but let me know what you had in mind!

Furthermore, as a daily sleap user (thank you very much for your work!) two very small GUI improvements could drastically improve my experience: 1- the ability to modify the preview of a video (eg, contrast, brightness) directly in the GUI (For now, I have to pre-process a large number of videos before using SLEAP just for brightness) and 2- in the 'run traning' box, the ability to customize where and how many predictions to make.

Yep, this is a great suggestion. I thought we had a thread for it but I can't seem to find it.

This is very achievable by just exposing the contrast/brightness/histogram in the GUI and letting the user adjust how images are displayed on the fly through the frame rendering framework.

Love it and hopefully we can add that one soon :)

Cheers,

Talmo

[Elleiv]

For the confidence score, I don't think it's very useful or practical to have a high gradient of confidence (at least for my own data, I know it is visible, invisible, or I don't know), so an intermediate score of 0.5 would be sufficient. For implementation in the GUI, if it is possible, I think a second right-click would be most practical (ie, first right-click: toggle visibility to 0, second right-click: uncertainty to 0.5).

Regarding the possibility of modifying light and contrast in video previews, your proposal seems perfect it, I can't wait to see the implementation (it'll save me hundreds of hours of preprocessing and backup space)!

Thanks a lor for your answer :)

Best