Paper | Video | Project Page
This is the official implementation of the CVPR 2021 submission LEAP: Learning Articulated Occupancy of People
LEAP is a neural network architecture for representing volumetric animatable human bodies. It follows traditional human body modeling techniques and leverages a statistical human prior to generalize to unseen humans.
If you find our code or paper useful, please consider citing:
@InProceedings{LEAP:CVPR:21,
title = {{LEAP}: Learning Articulated Occupancy of People},
author = {Mihajlovic, Marko and Zhang, Yan and Black, Michael J and Tang, Siyu},
booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2021},
}
Contact Marko Mihajlovic for questions or open an issue / a pull request.
Download a SMPL body model (SMPL, SMPL+H, SMPL+X, MANO) and store it under ${BODY_MODELS}
directory of the following structure:
${BODY_MODELS}
├── smpl
│ └── x
├── smplh
│ ├── male
| │ └── model.npz
│ ├── female
| │ └── model.npz
│ └── neutral
| └── model.npz
├── mano
| └── x
└── smplx
└── x
NOTE: currently only SMPL+H model is supported. Other models will be available soon.
Another prerequest is to install python packages specified in the requirements.txt
file, which can be conveniently
accomplished by using an Anaconda environment:
# clone the repo
git clone https://github.com/neuralbodies/leap.git
cd ./leap
# create environment
conda env create -f environment.yml
conda activate leap
and install the leap
package via pip
:
# note: install the build-essentials package if not already installed (`sudo apt install build-essential`)
python setup.py build_ext --inplace
pip install -e .
Download LEAP pretrained models from here and extract them under ${LEAP_MODELS}
directory.
Check demo code in examples/query_leap.py
for a demonstration on how to use LEAP for differentiable occupancy checks.
Follow instructions specified in data_preparation/README.md
on how to prepare training data.
Then, replace placeholders for pre-defined path variables in configuration files (configurations/*.yml
) and execute training_code/train_leap.py
script to train the neural network modules.
LEAP consists of two LBS networks and one occupancy decoder.
cd training_code
To train the forward LBS network, execute the following command:
python train_leap.py ../configurations/fwd_lbs.yml
To train the inverse LBS network:
python train_leap.py ../configurations/inv_lbs.yml
Once the LBS networks are trained, execute the following command to train the occupancy network:
python train_leap.py ../configurations/leap_model.yml
See specified yml configuration files for details about network hyperparameters.