There is a large discrepancy in the Visa dataset results

[jin-123-456]

category,AP-det,AP-loc,P-AUROC,I-AUROC,AUPRO,seg-AP-det,seg-I-AUROC
candle,0.9076238870620728,0.17265495657920837,0.963451623916626,0.8877000212669373,0.8993771076202393,0.9461877346038818,0.9323999881744385
capsules,0.6026958227157593,0.00510335061699152,0.6890470385551453,0.4351666569709778,0.45776668190956116,0.6549073457717896,0.5261666774749756
cashew,0.9606648683547974,0.5859807133674622,0.9902759790420532,0.9073999524116516,0.8819827437400818,0.9770940542221069,0.9484000205993652
chewinggum,0.7460821866989136,0.009644925594329834,0.7314893007278442,0.5291999578475952,0.26553836464881897,0.6337662935256958,0.3733999729156494
fryum,0.6968380212783813,0.07191576063632965,0.8529053330421448,0.46719998121261597,0.4841606020927429,0.6894668340682983,0.5289999842643738
macaroni1,0.44285985827445984,0.0005755668971687555,0.6816428303718567,0.41269999742507935,0.20409470796585083,0.46284759044647217,0.4788999855518341
macaroni2,0.5751076936721802,0.0002994732349179685,0.5580939054489136,0.5595999956130981,0.04122358560562134,0.5457344651222229,0.5827999711036682
pcb1,0.662883996963501,0.0069735487923026085,0.6827929615974426,0.6281000375747681,0.35054701566696167,0.5785677433013916,0.6351999640464783
This is part of the results I obtained from running the second dataset on my machine, and they differ quite a bit from the data in the paper. May I ask what could be the reason? I followed your steps—first ran prepare_visa.py to download the dataset, and then proceeded with train.py. Is this the correct procedure?

[jin-123-456]

Your project is truly excellent, but I noticed that for the grid category in the first dataset, my reproduced results differ significantly from those in the paper. What could be the reason for this discrepancy?

last.csv

[blaz-r]

Hi. I am not sure what causes this discrepancy; the results should match the ones in the paper pretty closely. There is a possibility that some runs could lead to slightly lower results. I think that the results for Grid are just due to an unlucky seed in the run. However, the gap on VisA is way too large. Did you change any code? Do you use the exact same versions of libraries? Could you also maybe share the loss for these runs?

[jin-123-456]

Thanks for the detailed explanation!

[whiterAutumn]

Hello, thank you very much for your work and sharing. However, I have a question that hope you can help me with. Are the indicators mentioned in the table at the pixel level or image level? I am looking forward to your reply.

[blaz-r]

Anomaly detection refers to image-level predictions, measured in AUROC. Anomaly localisation refers to pixel-level predictions, measured in AUPRO.