This repository contains the code of our CVPR 2021 work "MetaSAug: Meta Semantic Augmentation for Long-Tailed Visual Recognition".

Real-world training data usually exhibits long-tailed distribution, where several majority classes have a significantly larger number of samples than the remaining minority classes. This imbalance degrades the performance of typical supervised learning algorithms designed for balanced training sets. In this paper, we address this issue by augmenting minority classes with a recently proposed implicit semantic data augmentation (ISDA) algorithm, which produces diversified augmented samples by translating deep features along many semantically meaningful directions. Importantly, given that ISDA estimates the classconditional statistics to obtain semantic directions, we find it ineffective to do this on minority classes due to the insufficient training data. To this end, we propose a novel approach to learn transformed semantic directions with metalearning automatically. In specific, the augmentation strategy during training is dynamically optimized, aiming to minimize the loss on a small balanced validation set, which is approximated via a meta update step. Extensive empirical results on CIFAR-LT-10/100, ImageNet-LT, and iNaturalist2017/2018 validate the effectiveness of our method

If you find this idea or code useful for your research, please consider citing our paper:

@inproceedings{li2021metasaug,
  title={Metasaug: Meta semantic augmentation for long-tailed visual recognition},
  author={Li, Shuang and Gong, Kaixiong and Liu, Chi Harold and Wang, Yulin and Qiao, Feng and Cheng, Xinjing},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={5212--5221},
  year={2021}
}

• PyTorch >= 1.2.0
• Python3
• torchvision
• PIL
• argparse
• numpy

We provide several trained models of MetaSAug for evaluation.

Testing on CIFAR-LT-10/100:

• sh scripts/MetaSAug_CE_test.sh
• sh scripts/MetaSAug_LDAM_test.sh

Testing on ImageNet and iNaturalist18:

• sh ImageNet_iNat/test.sh

The trained models are in Google Drive.

• Long-tailed CIFAR10/100: The long-tailed version of CIFAR10/100. Code for coverting to long-tailed version is in data_utils.py.
• ImageNet-LT: The long-tailed version of ImageNet. [Long-tailed annotations]
• iNaturalist2017: A natural long-tailed dataset.
• iNaturalist2018: A natural long-tailed dataset.

Training on CIFAR-LT-10/100:

CIFAR-LT-100, MetaSAug with LDAM loss
python3.6 MetaSAug_LDAM_train.py --gpu 0 --lr 0.1 --lam 0.75 --imb_factor 0.05 --dataset cifar100 --num_classes 100 --save_name MetaSAug_cifar100_LDAM_imb0.05 --idx 1

Or run the script:

sh scripts/MetaSAug_LDAM_train.sh

Training on ImageNet-LT:

CUDA_VISIBLE_DEVICES=0,1,2,3 python3 -m torch.distributed.launch --nproc_per_node=4 --master_port 53212 train.py  --lr 0.0003 --meta_lr 0.1 --workers 0 --batch_size 256 --epochs 20 --dataset ImageNet_LT --num_classes 1000 --data_root ../ImageNet

Or run the script:

sh ImageNet_iNat/scripts/train.sh

Note: Training on large scale datasets like ImageNet-LT and iNaturalist2017/2018 involves multiple gpus for faster speed. To achieve better generalizable representations, vanilla CE loss is used for training the network in the early training stage. For convenience, the training starts from the pre-trained models, e.g., ImageNet-LT, iNat18 (both from project cRT).

CIFAR-LT-10

CIFAR-LT-100

ImageNet-LT

iNaturalist18

Some codes in this project are adapted from Meta-class-weight and cRT. We thank them for their excellent projects.