Project: ASL Classification Using CNNs
- Shalvi Saxena
- Vikram Singh Brahm
- Simran Sohal
- Nishant Saini
-
In Canada alone, there more than 357,000 people who're part of the Deaf Community and American Sign Language is the most tool for them for communication. In this study, we approach the problem of recognizing static hand gestures using Convolutional Neural Networks and recognize and help in interpretation of these hand gestures.
-
Proposed a systematic approach using ResNet18, MobileNetV3, and ShuffleNetV2 models trained on different facets of ASL i.e. punctuations, numbers and alphabets. We study, how different architecutres approach the problem and how we train these networks from scratch or pretrained weights and study the precisions, accuracy and f1 score and what makes networks better for training.
- PIL 9.2.0
- cv2 4.7.0
- matplotlib 3.5.2
- numpy 1.21.5
- sklearn 1.0.2
- torch 1.8.1+cu111
- torchvision 0.9.1+cu101
- cuda V11.1
- shutil 11.0.0
- To train and validate the model, we have created multiple .ipynb files for each dataset and each model. The models are saved in the respective folders.
- To train the model, change the location of the dataset in the code or link the kaggle account in your .ipynb files. The dataset can be downloaded automatically from the kaggle. Run all the blocks under the training section. The models weigh would be saved in the current directory.
- User can also run the validation models by running the blocks under validation section in the .ipynb files. Dataloader should be preloaded with the test dataset. The validation accuracy, precision, recall and f1 score would be printed and T-SNE plots would be generated.