Won 1st place AI/ML, overall 2nd place in Anaconda's Data Science Expo 2023 supported by AI Singapore
A tensorflow.js web application that utilies TF2 object detection models to recognize real-time static American Sign Language (ASL) via web browser. This web application comes in a form of a game that recognises ASL alphabets via the user's web cam. Try the live demo at https://learnsign.vercel.app.
Using the concept of transfer learning, we finetuned TensorFlow 2 Detection Model Zoo SSD MobileNetv2 FPNLite 320x320 model weights via the TF2 Object Detection API in Google Colab. As the computer vision models are running on tensorflow.js, inference is carried out on cilent side and no video/image data from the user is sent to the website hosting server.
- 4 Classes: A, B, C, D.
- Finetuned on SSD MobileNetv2 FPNLite 320x320 pre-trained on COCO 2017 dataset.
- 24 Classes: A, B, C, D, E, F, G, H, I, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y.
- J and Z are excluded as they are both dynamic sign langauges involving movement.
- Finetuned on SSD MobileNetv2 FPNLite 320x320 pre-trained on COCO 2017 dataset.
-
Test mean average precision (mAP) and average recall
-
Test Confusion Matrix @0.5IOU
-
Test Precision Recall Plot @0.5IOU
-
Test Precision Plot @0.5IOU
-
Test Recall Plot @0.5IOU
-
Inference on Test Image (Groundtruth bounding box in blue with IOU Score of prediction result)
