Scripts for evaluating results on the MPII human pose dataset. "Single person pose estimation" is the only type of evaluation currently supported.
Disclaimer: This is an unofficial repository, I am not from MPI and I was not involved in the creation of the dataset.
Predictions are expected to have the following format:
- Must be a Matlab (.mat) or HDF5 (.h5) file
- Must have one field,
preds
, which is the joint predictions tensor - Tensor size must be [2 x 16 x n] or [n x 16 x 2]
- Must have one field,
- Must correspond to one of the following subsets: train, val, test
- See annot/{train,valid,test}.h5 for which examples are in each of these subsets
Predictions produced by the following repositories meet these requirements:
- https://github.com/bearpaw/pytorch-pose
- https://github.com/anewell/pose-hg-demo
The PCKh performance metric is the percentage of joints with predicted locations that are no further than half of the head segment length from the ground truth.
"PCKh total" excludes the pelvis and thorax joints from the calculation, presumably because they are very easy to predict given that the approximate person center is provided.
Loads predictions from Matlab or HDF5 files and compares them with ground
truth labels to calculate accuracy metrics (eg PCKh). You will want to edit
getExpParamsNew.m
to add new sets of predictions, and evalMPII.m
to specify
which predictions to include and which subset (train/val) to use.
Loads flat test set predictions and prepares them for submission.
The preds/reference
directory contains multiple validation set prediction
files generated by established pose estimation models.
You can compare against these predictions using evalMPII.m
.
NOTE: Since the reference predictions are for the validation set, they are
not compatible with the prepareTestResults.m
script.
In order to keep evaluation in line with existing work, a lot of files in this repository were copied verbatim from other sources.
- https://datasets.d2.mpi-inf.mpg.de/andriluka14cvpr/mpii_human_pose_v1_u12_2.zip
annot/mpii_human_pose_v1_u12_1.mat
- https://human-pose.mpi-inf.mpg.de/results/mpii_human_pose/evalMPII.zip
- Contents of the
eval/
folder
- Contents of the
- https://github.com/anewell/pose-hg-train/tree/master/data/mpii/annot
annot/train.h5
annot/valid.h5
annot/test.h5