Official code of Transformer-based Unified Recognition of Two Hands Manipulating Objects
Hoseong Cho, Chanwoo Kim, Jihyeon Kim, Seongyeong Lee, Elkhan Ismayilzada, Seungryul Baek
| Input | Pose | Contact |
|---|---|---|
 |
 |
 |
 |
 |
 |
 |
 |
 |
git clone https://github.com/chohoseong/H2OTR.git
conda install pytorch==1.8.0 torchvision==0.9.0 torchaudio==0.8.0 cudatoolkit=11.1 -c pytorch -c conda-forge
pip install -r requirements.txt
conda install -c fvcore -c iopath -c conda-forge fvcore iopath
conda install pytorch3d -c pytorch3d
Compiling CUDA operators
cd ./models/ops
sh ./make.sh
Register and download MANO model.
.H2OTR
├── AIK
│ ├── AIK_config.py
│ └── AIK_torch.py
├── config
│ ├── H2O
│ └── FPHA
├── mano
│ ├── models
│ └── ...
├── manopth
│ ├── manolayer.py
│ └── ...
├── models
│ ├── H2OTR.py
│ └── ...
├── README.md
├── demo.py
├── requirements.txt
...
.H2OTR
└── data
├── FPHA
│ ├── Object_6D_pose_annotation_v1
│ ├── Video_files
│ ├── data_split_action_recognition.txt
│ └── ...
└── H2O
├── action_labels
├── subject1
├── subject2
└── ...
You can access the pretrained weight file for the H2O and FPHA datasets by following the link
python demo.py \
--dataset_file H2O \
--pretrained_model weights/H2O.pth \
--vid_id 9
@inproceedings{cho2023transformer,
title={Transformer-Based Unified Recognition of Two Hands Manipulating Objects},
author={Cho, Hoseong and Kim, Chanwoo and Kim, Jihyeon and Lee, Seongyeong and Ismayilzada, Elkhan and Baek, Seungryul},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={4769--4778},
year={2023}
}