This is the official repository of the paper: Efficient People Counting in Thermal Images: The Benchmark of Resource-Constrained Hardware.
The repository includes the code for reproducing experiments and obtaining results from the aforementioned publication. This repository is organized as follows:
- thermo directory contains code and scripts related to model training, optimization, and quantization phases,
- evaluation folder consists of hardware deployment, inference, and evaluation tools.
@ARTICLE{9964383,
author={Piechocki, Mateusz and Kraft, Marek and Pajchrowski, Tomasz and Aszkowski, Przemyslaw and Pieczynski, Dominik},
journal={IEEE Access},
title={Efficient People Counting in Thermal Images: The Benchmark of Resource-Constrained Hardware},
year={2022},
volume={10},
number={},
pages={124835-124847},
doi={10.1109/ACCESS.2022.3225233}
}
The benchmark utilizes Thermo Presence dataset that consists of 13 644 low-resolution thermal images recorded in office spaces. The dataset distribution is shown in the table below. The training, validation, and test split was chosen according to the authors' choice which is described in their project repository.
0 | 1 | 2 | 3 | 4 | 5 | Total | |
---|---|---|---|---|---|---|---|
Training | 99 | 105 | 2984 | 3217 | 1953 | 114 | 8472 |
Validation | 0 | 139 | 631 | 1691 | 225 | 139 | 2825 |
Test | 162 | 83 | 211 | 341 | 1235 | 315 | 2347 |
Device | Target Hardware | Max Clock Frequency | FLASH | SRAM | Evaluation Framework |
---|---|---|---|---|---|
Arduino Nano 33 BLE Sense | nRF52840 | 64 MHz | 1 MB | 256 KB | TFLite Micro |
Arduino Portenta H7 | STM32H747 | 480 MHz | 2 MB | 1 MB | TFLite Micro |
LOLIN32 | ESP32-WROOM-32 | 240 MHz | 4 MB | 520 KB | TFLite Micro with ESP-NN support |
STM32 F429ZI Nucleo-144 | STM32F429 | 180 MHz | 2 MB | 256 KB | STM32 Cube AI |
STM32 H745ZI Nucleo-144 | STM32H745 | 480 MHz | 2 MB | 1 MB | STM32 Cube AI |
Raspberry Pi 4B | Quad Core Cortex-A72 | 1.5 GHz | * | 2 GB | TensorFlow Lite |
Coral USB Accelerator | Google Edge TPU | 500 MHz | * | 8 MB | TensorFlow Lite |
Intel Neural Compute Stick 2 | Intel Movidius Myriad X VPU | 700 MHz | * | 4 GB | OpenVINO |
The neural network is based on U-Net architecture with shallow structure, single-channel input, and output. The proposed model has only 46 577 parameters.
Metric Name | MAE | MSE | Counting MAE | Counting MSE | Counting MRAPE [%] | Accuracy | F1 Score | No. of parameters |
---|---|---|---|---|---|---|---|---|
Results | 0.1057 | 0.0332 | 0.0226 | 0.0234 | 0.81 | 0.9778 | 0.9782 | 46 577 |
Device | Data Type | Avg. Inference Time [ms] | Counting MAE | Counting MSE | Counting MRAPE [%] |
---|---|---|---|---|---|
Arduino Nano 33 BLE Sense | INT8 | 1430.125 ±1.143 | 0.023 | 0.024 | 0.82 |
Arduino Portenta H7 | INT8 | 137.494 ±0.500 | 0.023 | 0.024 | 0.82 |
LOLIN32 | INT8 | 840.442 ±0.021 | 0.023 | 0.023 | 0.81 |
STM32 F429ZI Nucleo-144 | INT8 | 230.939 ±0.100 | 0.036 | 0.038 | 1.14 |
STM32 H745ZI Nucleo-144 | FP32 | 165.983 ±0.100 | 0.023 | 0.023 | 0.81 |
INT8 | 53.260 ±0.100 | 0.036 | 0.038 | 1.14 | |
Raspberry Pi 4B | FP32 | 7.707 ±0.591 | 0.023 | 0.023 | 0.81 |
FP16 | 7.690 ±0.558 | 0.023 | 0.023 | 0.81 | |
INT8 | 4.194 ±0.052 | 0.038 | 0.039 | 1.18 | |
Raspberry Pi 4B + Coral USB Accelerator (std) | INT8 | 0.605 ±0.044 | 0.037 | 0.038 | 1.16 |
Raspberry Pi 4B + Coral USB Accelerator (max) | INT8 | 0.570 ±0.060 | 0.037 | 0.038 | 1.16 |
Raspberry Pi 4B + Intel Neural Compute Stick 2 | FP32 | 2.630 ±0.159 | 0.028 | 0.029 | 0.92 |
FP16 | 2.300 ±0.100 | 0.028 | 0.029 | 0.91 |
The figure above consists of current consumption characteristics measured on benchmarked hardware. The graphs are limited to 180 seconds in order to highlight peaks related to the prediction stage.