Comparative analysis of neural network models performance on low-power devices for a real-time object detection task

A computer vision based real-time object detection on low-power devices is economically attractive, yet a technically challenging task. The paper presents results of benchmarks on popular deep neural network models, which are often used for this task. The results of experiments provide insights into trade-offs between accuracy, speed, and computational efficiency of MobileNetV2 SSD, CenterNet MobileNetV2 FPN, EfficientDet, YoloV5, YoloV7, YoloV7 Tiny and YoloV8 neural network models on Raspberry Pi 4B, Raspberry Pi 3B and NVIDIA Jetson Nano with TensorFlow Lite. We fine-tuned the models on our custom dataset prior to benchmarking and used post-training quantization (PTQ) and quantization-aware training (QAT) to optimize the models’ size and speed. The experiments demonstrated that an appropriate algorithm selection depends on task requirements. We recommend EfficientDet Lite 512$\times$512 quantized or YoloV7 Tiny for tasks that require around 2 FPS, EfficientDet Lite 320$\times$320 quantized or SSD Mobilenet V2 320$\times$320 for tasks with over 10 FPS, and EfficientDet Lite 320$\times$320 or YoloV5 320$\times$320 with QAT for tasks with intermediate FPS requirements.