unexpected keys in pre-trained model loading

[khanhha]

Hello, thank you very much for creating a very helpful pose estimation project.
I am trying to fine-tune the model configs/top_down/hrnet/coco/hrnet_w48_coco_384x288.py with the pre-trained weights: hrnet_w48_coco_384x288-314c8528_20200708.pth on the posetrack dataset, and the below warning show up the console.

The training command line I used is:
python tools/train.py ./configs/top_down/hrnet_mvai/coco/hrnet_w48_coco_384x288.py

And here is the pre-trained weight that I used.

And here is the warning message:

unexpected key in source state_dict: backbone.conv1.weight, backbone.bn1.weight, backbone.bn1.bias, backbone.bn1.running_mean, backbone.bn1.running_var, backbone.bn1.num_batches_tracked, backbone.conv2.weight, backbone.bn2.weight, backbone.bn2.bias, backbone.bn2.running_mean, backbone.bn2.running_var, backbone.bn2.num_batches_tracked, backbone.layer1.0.conv1.weight, backbone.layer1.0.bn1.weight, etc

missing keys in source state_dict: conv1.weight, bn1.weight, bn1.bias, bn1.running_mean, bn1.running_var, conv2.weight, bn2.weight, bn2.bias, bn2.running_mean, bn2.running_var, layer1.0.conv1.weight, layer1.0.bn1.weight, layer1.0.bn1.bias, layer1.0.bn1.running_mean, layer1.0.bn1.running_var, layer1.0.conv2.weight, layer1.0.bn2.weight, layer1.0.bn2.bias, layer1.0.bn2.running_mean, layer1.0.bn2.running_var, layer1.0.conv3.weight, layer1.0.bn3.weight, layer1.0.bn3.bias, layer1.0.bn3.running_mean, layer1.0.bn3.running_var, layer1.0.downsample.0.weight, layer1.0.downsample.1.weight, layer1.0.downsample.1.bias, layer1.0.downsample.1.running_mean, layer1.0.downsample.1.running_var, layer1.1.conv1.weight, layer1.1.bn1.weight, layer1.1.bn1.bias, layer1.1.bn1.running_mean, layer1.1.bn1.running_var, layer1.1.conv2.weight, layer1.1.bn2.weight, layer1.1.bn2.bias, layer1.1.bn2.running_mean, layer1.1.bn2.running_var, layer1.1.conv3.weight, layer1.1.bn3.weight, etc

[khanhha]

UPDATE:

I have changed the pretrained field in model settings in configs/top_down/hrnet/coco/hrnet_w48_coco_384x288.py using the URL "https://open-mmlab.s3.ap-northeast-2.amazonaws.com/pretrain/third_party/hrnetv2_w48-d2186c55.pth", but the error still occurs.

I debugged the function checkpoint.py/load_state_dict, and the list missing_keys is now empty, but the unexpected_keys is still not empty.

Could you please help me explain this problem? Should I revert to an old version of the repos to get compatibility?

[jin-s13]

We did not expect the user to load a COCO trained model as pretrained. Currently, it only supports using imagenet-pretrained model.
We will fix this part today.

[Canwang-sjtu]

First, did you modify the model structure?
Second, in the second line of configs/top_down/hrnet/coco/hrnet_w48_coco_384x288.py, you may use load_from = hrnet_w48_coco_384x288-314c8528_20200708.pth and set pretrained=None in line 42

[jin-s13]

UPDATE:

I have changed the pretrained field in model settings in configs/top_down/hrnet/coco/hrnet_w48_coco_384x288.py using the URL "https://open-mmlab.s3.ap-northeast-2.amazonaws.com/pretrain/third_party/hrnetv2_w48-d2186c55.pth", but the error still occurs.

I debugged the function checkpoint.py/load_state_dict, and the list missing_keys is now empty, but the unexpected_keys is still not empty.

Could you please help me explain this problem? Should I revert to an old version of the repos to get compatibility?

Don't worry, it is normal to have unexpected_keys for HRNet. Please just ignore it.

[khanhha]

Fantastic!

Thank you very much for your support.

When I set pretrained to None, and set load_from to point to "hrnet_w48_coco_384x288-314c8528_20200708.pth". The pre-trained model is smoothly. missing_keys and unexpected_keys are empty now.

I hope that you add pre-training from COCO soon. It would be tremendously helpful if we can fine-tune from pre-trained model on COCO.

Again, thank you for your valuable works. The code is very well-organized and clean, the supported models are diverse, the model zoo is awesome. This is the best pose open-source project I have came across.

Best.

[khanhha]

PS. I don't change the model architecture. Just the path in the data-settings, and data_cfg.use_gt_bbox to True because PoseTrack doesn't have bb detections.