ILSVRC2012 ImageNet 10K Validataion Images and devkit for Top-1 & Top-5 Accuracy Benchmarking in Edge Devices [example: RPi3/4]
git clone https://github.com/ghimiredhikura/ILSVRC2012_img_val_10K
cd ILSVRC2012_img_val_10K/data/
unzip mobilenet_v2_imagenet.zip
rm -rf mobilenet_v2_imagenet.zip
mv mobilenet_v2_imagenet.txt ../mobilenet_v2_imagenet.txt
cd ../
Download ILSVRC2012 validataion images - 10K images and use your inference engine to get dump file. Example: inference script, reference models running scipt.
chmod +x scripts/download_ILSVRC2012_img_val_10K.sh
./scripts/download_ILSVRC2012_img_val_10K.sh
Now use your prediction dump file generated using your favorite inference engine to get top-1 and top-5 accuracies. Here we provide an example imagenet dump file on 10K images using opencv-4.1.1 dnn classification engine with mobilenetv2 caffe model. Once you have this dump file you can use following command to get top-1 and top-5 accuracies.
source run_nvisoeval.sh mobilenet_v2_imagenet.txt
| Inference Engine | Model | Valid Images | RaspberryPi4B-armv7l | RaspberryPi4B-armv7l | RaspberryPi4B-aarch64 | RaspberryPi4B-aarch64 | Issue Discussion |
|---|---|---|---|---|---|---|---|
| Top-1 Accuracy | Top-5 Accuracy | Top-1 Accuracy | Top-5 Accuracy | ||||
| opencv_4.1.1 | Mobilenet_V2_1.0_224_caffe | 10K | 62.32% | 84.04% | click | ||
| tflite_2.0.0 | Mobilenet_V2_1.0_224_tflitehm | 10K | 71.06% | 90.10% |