Official PyTorch implementation of "RenderIH: A large-scale synthetic dataset for 3D interacting hand pose estimation", ICCV 2023 Project website
RenderIH: Download from Google Drive: imgs, annotations, materials; or BaiduPan: imgs annotations. Untar the compressed files of imgs and annotations, then run step7 in rendering_code. Materials is used for generation process in previous steps in rendering.
download and unzip [misc.tar].
Register and download MANO data. Put MANO_LEFT.pkl
and MANO_RIGHT.pkl
in misc/mano
After collecting the above necessary files, the directory structure of ./misc
is expected as follows:
./misc
├── mano
│ └── MANO_LEFT.pkl
│ └── MANO_RIGHT.pkl
├── model
│ └── config.yaml
├── graph_left.pkl
├── graph_right.pkl
├── upsample.pkl
├── v_color.pkl
- Tested with python3.8.8 on Ubuntu 18.04, CUDA 11.3.
torch1.12.1: pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cu113
pytorch3d: pip install fvcore iopath; pip install git+https://github.com/facebookresearch/pytorch3d.git@stable
opencv4.7:pip install opencv_python==4.7.0.72
manopth pip install git+https://github.com/hassony2/chumpy.git
,pip install git+https://github.com/hassony2/manopth.git
"sdf" change AT_CHECK in multiperson/sdf/csrc/sdf_cuda.cpp
to TORCH_CHECK
mmcv:pip install -U openmim
,mim install mmcv
numpy,tqdm,yacs==0.1.8,tensorboardX,scipy,imageio,matplotlib,scikit-image,manopth,timm,imgaug,fvcore,iopath
-
Download InterHand2.6M dataset and unzip it. (Noted: we used the v1.0_5fps version and H+M subset for training and evaluating)
-
Process the dataset by :
python utils/dataset_gen/interhand.py --data_path PATH_OF_INTERHAND2.6M --save_path ./interhand2.6m/ --gen_anno 1
python utils/dataset_gen/interhand.py --data_path ./interhand2.6m/ --gen_anno 0
Replace PATH_OF_INTERHAND2.6M with your own store path of InterHand2.6M dataset.
-
Download Hand-Hand Interaction from the website, categories from Walking to Hugging (01.zip~07.zip). Moreover, download the mano annotations from MANO_fits.
-
Process the dataset by:
python utils/dataset_gen/tzionas_generation.py --mano_annot xxx --detection_path xxx --rgb_path xxx --output_path xxx
model without syntheic data model with synthetic data
python apps/train.py --gpu 0,1,2,3
change INTERHAND_PATH
in utils/default.yaml
to the dataset path
utils/default.yaml
has some argments that can be tuned
python apps/eval_interhand.py --model MODEL_PATH --data_path INTERHAND2.6M_PATH
change MODEL_PATH
to the pretrained model path, and INTERHAND2.6M_PATH
to dataset path.
data_type=0, dataset/interhand.py syn=True, use renderih together with Interhand2.6M
data_type=1, loader_ori using synthetic+real
data_type=2, loader.py using interhand_withother.py, training ego3dhand , h2o3d,or renderih
data_type=3, loader.py, using interhand_orisyn.py ,using the synthetic data
data_type=4, loader.py, using interhand_subset.py ,poseaug, subset synthetic and full real interhand data
utils/compute_maskiou.py
. Calculate the iou distribution for hand data.
@inproceedings{li2023renderih,
title={Renderih: A large-scale synthetic dataset for 3d interacting hand pose estimation},
author={Li, Lijun and Tian, Linrui and Zhang, Xindi and Wang, Qi and Zhang, Bang and Bo, Liefeng and Liu, Mengyuan and Chen, Chen},
booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
pages={20395--20405},
year={2023}
}