Skip to content

lucamagnasco/AI-Dance-based-on-Human-Pose-Estimation

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

10 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

AI-Dance-based-on-Human-Pose-Estimation

A Human Pose Skeleton represents the orientation of a person in a graphical format. Essentially, it is a set of coordinates that can be connected to describe the pose of the person. Each co-ordinate in the skeleton is known as a part (or a joint, or a keypoint). A valid connection between two parts is known as a pair (or a limb).

Dependencies

Flow of Human Pose Estimation

  • Extraction of pose coordinates from dance videos using openpose human pose estimation.
  • Training LSTM network on extracted coordinates using video as input and coordinates as output.
  • Display output videos by joining predicted coordinates to generate dancing human stick figures.

How to Execute Code

  1. You will first have to download the repository and then extract the contents into a folder.
  2. Make sure you have the correct version of Python installed on your machine. This code runs on Python 3.6 above.
  3. Now, install the required libraries.
  4. Now go to src folder and run extract_data.py to download videos and audios to data folder. You can add youtube videos links to "video_links.txt" file for downloading.
  5. Download pretrained weights for pose estimation from here. Download pose_iter_440000.caffemodel and save it in "models" folder.
  6. Run demo.py to train LSTM and display predicted dance video.

python main.py --video data/video.mp4 --audio data/audio.wav --background data/bg1.jpg

Results

  1. Pose estimation using openpose on image

image

  1. Pose estimation using openpose on video

NOte : For more information about implementation details, please go through AI Dance based on Human Pose Estimation

References

  1. OpenPose
  2. Moviepy
  3. Pytube
  4. Dancing AI

About

Human Pose Estimation using Deep Learning.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 100.0%