This project is about using Deep Learning algorithms in order to build an AI software to recognize day to day human activities or actions. We are using state of the art Computer Vision models like InceptionV3 to extract key features from videos frames and trained a sequence model classifier to recognize a set of actions or activities choosen upfront. We used this paper as an inspiration.
We have use a subset (10 actions) of UCF101. Our computer vision model has been trained to be able to recognize the following action:
- ApplyEyeMakeup
- BabyCrawling
- Biking
- Billiards
- BrushingTeeth
- PizzaTossing
- PushUps
- Swing
- TrampolineJumping
- Typing
The following pictures explains how our models takes in raw video data and output its associated activity or action label.
In this project, we are handling videos, which pose some challenges in terms of compution especially when reading videos, capturing frames and extracting features via a CNN.
We tried to leverage the power of parrallel computing when running our code so that everything runs as fast as possible on multiple CPU cores via multithreading.
Note: If possible, run this project on a GPU powered environment for faster computations.
-
Clone this repo (for help see this tutorial).
-
You can download raw data from here Raw Data is being kept here .
Choose the activities videos folders you're interested in (or everything) and put them inside the dataset folder Note: Be aware that the whole dataset of UCF is about 6.5Go
-
Create a virtual env in the project folder (for help see this tutorial)
-
Run the following command to install the necessary packages
- For linux users:
pip3 install -r requirements.txt
- For windows users:
pip install -r requirements.txt
- Open the Har.ipynb notebook and run the cells.
Please choose the python virtual environment you created previously