MPM: A Unified 2D-3D Human Pose Representation via Masked Pose Modeling
Zhenyu Zhang*, Wenhao Chai*, Zhongyu Jiang, Tian Ye, Mingli Song, Jenq-Neng Hwang, Gaoang Wang
arXiv 2023.
pip install torch matplotlib
Our model is evaluated on Human3.6M and MPI-INF-3DHP datasets, and we ese AMASS dataset for better pre-training.
Dataset setting is same as this repo P-STMO. You can download the processed .npz file from their repo and put the .npz files in ./dataset folder.
coming soon
Model checkpoint is not published yet.
python trainer.py -f 243 --n_joints 17 --gpu 0,1 --reload 1 --layers 4 -tds 2 --previous_dir x.pth --refine --refine_reload 1 x_refine.pth
python trainer.py -f 243 k gt --n_joints 17 --gpu 0,1 --reload 1 --layers 4 -tds 2 --previous_dir x.pth --refine --refine_reload 1 --previous_refine_name x_refine.pth
python trainer_3dhp.py -f 243 --n_joints 16 --gpu 0,1 --reload 1 --layers 4 -tds 1 --previous_dir x.pth --refine --refine_reload 1 --previous_refine_name x.pth
You should follow the instructions in poseaug and got generator checkpoint for human3.6M. Then put the generator checkpoints in ./Augpart/chk foler. You can put as many as you can get and modify the list in file ./Augpart/gan_preparation.py
python pretrainer.py --MAE -f 243 --train 1 -k gt --n_joints 17 -b 1024 -tds 2 --layers 4 --dataset h36m --lr 0.0001 -lrd 0.97 -tmr 0.6 -smn 5 --gpu x,y --name task_name python pretrainer.py --MAE -f 243 --train 1 -k gt --n_joints 16 -b 1024 -tds 2 --dataset h36m --lr 0.0001 -lrd 0.97 --layers 4 -tmr 0.6 -smn 5 --gpu x,y --name task_name python pretrainer.py --MAE -f 243 --n_joints 16 -b 1024 -k gt -tds 2 --train 1 --dataset h36m,3dhp,amass --layers 3 --lr 0.0001 -lrd 0.97 -tmr 0.6 -smn 5 --gpu x,y --name task_name N_JOINTS x and Layers n hould keep consistent with the pre-trained model.
python trainer.py -f 243 -k gt --train 1 --n_joints x -b 1024 --gpu 0,1 --lr 0.0007 -lrd 0.97 --layers 4 -tds 2 (--MAEreload 1 --previous_dir /path/to/pretrainedcheckpoint)(optional)After training on human3.6M dataset, you can refine the model by:
python trainer.py -f 243 -k gt --train 1 --n_joints x -b 1024 --gpu 0,1 --lr 0.0001 -lrd 0.97 --layers 4 -tds 2 --reload 1 --previous_dir /path/to/bestcheckpoint --refineFinetune 3DHPE Model for MPI_INF_3DHP with 16 joints:
python trainer_3dhp.py -f 243 -k gt --train 1 --n_joints 16 -b 1024 --gpu 0,1 --lr 0.0007 -lrd 0.97 --layers 3 -tds 1 (--MAEreload 1 --previous_dir /path/to/pretrainedcheckpoint)(optional)You can reload pretrained model or train model without reloading checkpoint:
python pretrainer.py -f 27 -b 2048 --model MAE -k gt --train 1 --layers 3 -tds 2 --lr 0.0002 -lrd 0.97 --name maskedliftcam -tmr 0 -smn 6 --gpu 0,1 --dataset h36m --MAE --comp2dlift 1
(--MAEreload 1 --MAEcp /path/to/model)(optional) You can reload pretrained model or train model without reloading checkpoint:
python pretrainer.py -f 27 -b 2048 --model MAE -k gt --train 1 --layers 3 -tds 2 --lr 0.0002 -lrd 0.97 --name comp3dcam -tmr 0 -smn 3 --gpu 0,1 --dataset h36m --MAE --comp3d 1
(--MAEreload 1 --MAEcp /path/to/model)(optional)Our code refers to the following repositories.
We thank the authors for releasing their codes.