Skip to content
/ DHM Public

[CVPR 2020] Dynamic Hierarchical Mimicking Towards Consistent Optimization Objectives

License

Notifications You must be signed in to change notification settings

d-li14/DHM

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

9 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Dynamic Hierarchical Mimicking

Official implementation of our DHM training mechanism as described in Dynamic Hierarchical Mimicking Towards Consistent Optimization Objectives (CVPR'20) by Duo Li and Qifeng Chen on CIFAR-100 and ILSVRC2012 benchmarks with the PyTorch framework.

We dissolve the inherent defficiency inside the deeply supervised learning mechanism by introducing mimicking losses between the main branch and auxiliary branches located in different network depths, while no computatonal overheads are introduced after discarding those auxiliary branches during inference.

Requirements

Dependencies

  • PyTorch 1.0+
  • NVIDIA-DALI (in development, not recommended)

Dataset

Download the ImageNet dataset and move validation images to labeled subfolders. To do this, you can use the following script: https://raw.githubusercontent.com/soumith/imagenetloader.torch/master/valprep.sh

ImageNet Pretrained models

Architecture Method Top-1 / Top-5 Error (%)
ResNet-18 DSL 29.728 / 10.450
DHM 28.714 / 9.940
ResNet-50 DSL 23.874 / 7.074
DHM 23.430 / 6.764
ResNet-101 DSL 22.260 / 6.128
DHM 21.348 / 5.684
ResNet-152 DSL 21.602 / 5.824
DHM 20.810 / 5.396

* DSL denotes Deeply Supervised Learning

* DHM denotes Dynamic Hierarchical Mimicking (ours)

Usage

Training

Please refer to the training recipes for illustrative examples .

Inference

After downloading the pre-trained models, you could test the model on the ImageNet validation set with this command line:

python imagenet.py \
    -a <resnet-model-name> \
    -d <path-to-ILSVRC2012-data> \
    --weight <pretrained-pth-file> \
    -e

Citations

If you find our work useful in your research, please consider citing:

@InProceedings{Li_2020_CVPR,
author = {Li, Duo and Chen, Qifeng},
title = {Dynamic Hierarchical Mimicking Towards Consistent Optimization Objectives},
booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2020}
}

Releases

No releases published

Packages

No packages published

Languages