PyTorch implementation of paper "Fast Emotion Recognition Neural Network for IoT Devices" by S. Mikhaylevskiy, V. Chernyavskiy and V. Pavlishen.
This project is focused on developing an efficient system for recognizing human emotions from facial expressions, designed to run on edge devices, with the final optimized models weighing just 6MB and achieving 4 FPS on Raspberry Pi 3. Pipeline consist of two separate models: one for face detection and the other for emotion classification. We could utilize one composite model, but...
# clone project
git clone https://github.com/lqrhy3/emotion-recognition.git
cd emotion-recognition
# [OPTIONAL] create conda environment
conda create -n emotion-recognition
conda activate emotion-recognition
# install requirements
pip install -r requirements.txt
# set PYTHONPATH variable
export PYTHONPATH=$PYHONPATH:/<path>/emotion-recognitionWhile the origins of the data remain unknown, both the training data and data for quantization are available on this repository. The dataset includes four categories of data: detection training data, detection calibration data, classification training data, and classification calibration data.
- Detection training data: Link
- Detection calibration data: Link
- Classification training data: Link
- Classification calibration data: Link
Training face-detection model:
python3 training/detection_train.py [-h] [--num_epochs EPOCHS] [--batch_size BATCH_SIZE] [--lr LEARNING RATE]
[--grid_size GRID_SIZE] [--num_bboxes NUM_BBOXES] [--val_split VAL_SPLIT]
[--path_to_log PATH_TO_LOG] [--model MODEL] [--path_to_data PATH_TO_DATA]Evaluating face-detection model:
python3 testing/detection_pipeline.py [-h] [--path_to_model] [--grid_size GRID_SIZE] [--num_bboxes NUM_BBOXES]
[--image_size IMAGE_SIZE] [--detection_threshold DETECTION_THRESHOLD] [--device DEVICE]Training emotion-classification model:
python3 training/classification_train.py [-h] [--num_epochs NUM_EPOCHS] [--batch_size BATCH_SIZE] [--lr LR]
[--val_split VAL_SPLIT] [--path_to_log PATH_TO_LOG] [--comment COMMENT]
[--model {MiniXception,ConvNet}] [--path_to_data PATH_TO_DATA] [--emotions EMOTIONS_LIST]Evaluating emotion-classification model:
python3 testing/classification_pipeline.py [-h] [--path_to_model PATH_TO_MODEL] [--image_size IMAGE_SIZE]
[--emotions EMOTIONS_LIST] [--device {cpu,cuda:0}]Running emotion-recognition pipeline:
python3 testing/detection&classification_pipeline.py [-h] [--path_to_detection_model PATH_TO_DETECTION_MODEL]
[--path_to_classification_model PATH_TO_CLASSIFICATION_MODEL]
[--grid_size GRID_SIZE] [--num_bboxes NUM_BBOXES] [--detection_size DETECTION_SIZE]
[--classification_size CLASSIFICATION_SIZE] [--detection_threshold DETECTION_THRESHOLD]
[--emotions EMOTIONS_LIST] [--device {cpu,cuda:0}]Running post-training quantization:
# detection model
python3 quantization/detection_post_training_quatization.py [-h] [--path_to_model PATH_TO_MODEL] [--path_to_data PATH_TO_DATA]
[--image_size IMAGE_SIZE] [--batch_size BATCH_SIZE] [--num_batches NUM_BATCHES]
[--save_type SAVE_TYPE]
# classification model
python3 quantization/classification_post_training_quatization.py [-h] [--path_to_model PATH_TO_MODEL] [--path_to_data PATH_TO_DATA]
[--image_size IMAGE_SIZE] [--batch_size BATCH_SIZE] [--num_batches NUM_BATCHES]
[--emotions EMOTIONS_LIST] [--save_type SAVE_TYPE]Running Flask app for demonstrating pipeline:
cd video-streaming
sudo chmod 0777 app.sh
./app.shYou should specify in config.ini path to quantized model
Application will be accessible at http:https://127.0.0.1:5000/
If you find our code, data or paper helpful, please consider citing:
@article{9444517,
author={Mikhaylevskiy, S. and Chernyavskiy, V. and Pavlishen, V. and Romanova, I. and Solovyev, R.},
booktitle={2021 International Seminar on Electron Devices Design and Production (SED)},
title={Fast Emotion Recognition Neural Network for IoT Devices},
year={2021},
volume={},
number={},
pages={1-6},
doi={10.1109/SED51197.2021.9444517}
}
This project was completed by Stanislav Mikhaylevskiy, Victor Pavlishen and Vladimir Chernyavskiy. If you have any questions or suggestions regarding this project, please feel free to contact us.