Face detection/recognition has been the most popular deep learning projects/researches for these past years. One of its daily application is the face verification feature to perform tasks on our devices (e.g., unlocking the device, signing in to some apps, confirming our payment, etc). However, this method could be prone to spoof attacks, in which the model could be fooled with the facial photograph of its respective user (i.e., using a printed or digital picture of the user, and many others face-spoofing attacks). Therefore, a facial anti-spoofing detection would be worth to develop for tackling this malicious problem.
This is the final machine learning project assignment of the Bangk!t program by Google, an exclusive machine learning academy led by Google, in collaboration with several Indonesian unicorn startups.
Due to the limited training resources (e.g., low computability and limited datasets), we divided the model into 2 models (detector and classifier) with sequential pipeline, as the following:
- Single Shot Multibox Detector (SSD), with the pretrain face detection model, as the detector.
- MobileNetV2, with transfer learning, as the classifier, trained using Kaggle notebook.
Note: To simplify the problem, we used the built-in models that are available on OpenCV and TensorFlow Keras respectively.
In this case, we only trained the classifier model and used the detector directly on the inference stage. Moreover, a complete pipeline of facial anti-spoofing detector is recommended fore the future improvements, although it requires a new dataset that also provides non-close-up images (i.e., full or half body) with the corresponding bounding box or facial key points label.
To limit our scope of work, we decided to tune the optimizer hyperparameter only (e.g., learning rate, scheduler, etc) and the class_weight as it's the one that arguably impacts the performance the most.
The dataset is highly imbalanced towards the spoof image, hence making class weight as an important training feature to avoid overfitting.
Install the dependencies from the requirements.txt
pip install -r requirements.txt
docs--- supporting documentations- Reference papers.
- Presentation slides.
input--- input directoriesdemo--- test case (image and video format).LCC_FASD--- Face anti-spoofing dataset (Timoshenko, et al. 2019).
output--- training results storage (e.g., trained weights, training history, etc)pretrain--- pretrain and trained model storage for inferencetrain--- training notebooks directory
This program uses a Command Line Interface (CLI) to run. There are 2 types of input option for the inference, detect as image or video. Write the following commands for either types.
Importing image from disk.
python3 inference.py \
--classifier ./pretrain/classifier/classifier.hdf5 \
--detector ./pretrain/detector \
--path ./input/demo/highres.jpg \
--image --resize 224 224 \
--save ./output/demo/highres_pred.jpg
Importing video from disk.
python3 inference.py \
--classifier ./pretrain/classifier/classifier.hdf5 \
--detector ./pretrain/detector \
--path ./input/demo/lowres.mp4 \
--video --resize 224 224 \
--save ./output/demo/lowres_pred.avi
For video input, WebCam streaming is also available by defining the camera ID as the input path instead.
python3 inference.py \
--classifier ./pretrain/classifier/classifier.hdf5 \
--detector ./pretrain/detector \
--path 0 \
--video --resize 224 224 \
--save ./output/demo/stream_pred.avi
- Single Shot Multibox Detector (Liu, et al., 2015); using the OpenCV's built-in SSD face detection model.
- MobileNetV2 (Sandler, et al., 2019)
This project is licensed under the MIT License - see the LICENSE.md file for details
- Liveness detection with OpenCV tutorial by Adrian Rosebrock.
- Face anti-spoofing open dataset by Timoshenko, et al. 2019.