Issues related to the data output/analysis

[jin-123-456]

This is a great project, but I have a question about the output - does the saved result always come from the final iteration? Shouldn't we typically expect the best-performing data to be saved instead?

[blaz-r]

Hi. Most of the anomaly detection methods use the output from the last epoch. While it is possible to save the best result, this best must be on the validation set. Since datasets like MVTec AD and VisA don't have a validation set, some started to use the best result on the test set. I think this is not a good practice as it is an explicit way of tuning on the test set. This is also the reason why it's not integrated here. If you have a validation set, the code can be easily modified to save the best weights on the validation set to be then used when testing. Hope this clears things up 😄

[jin-123-456]

The paper reports the average performance over 5 runs with different seeds. Could you specify what those 5 different seeds were? It would be greatly appreciated if you could share them. Thank you very much!

[blaz-r]

I'm not 100% sure, but in latest script I got these: (42 1337 420 0 456654)