Official MXNet implementation of Embedding Expansion: Augmentation in Embedding Space for Deep Metric Learning (CVPR 2020)
Byungsoo Ko*, Geonmo Gu* (* Authors contributed equally.)
@NAVER/LINE Vision
- Paper | Supplementary Video | PPT | Poster
Embedding Expansion (EE) is a novel augmentation method in embedding space which is compatible with existing pair-based metric learning losses.
By generating synthetic points with linear interpolation, they can contribute to training a network with augmented information.
Clustering and retrieval performance (%) on three benchmarks in comparison with other methods. † denotes the HPHN triplet loss, and bold numbers indicate the best score within the same loss.
$ pip3 install -r requirements.txt
- Download CARS196 dataset.
$ mkdir -p data/CARS_196
$ wget http:https://ai.stanford.edu/~jkrause/car196/car_ims.tgz
$ tar -xvzf car_ims.tgz -C data/CARS_196
- Available losses: HPHN-triplet, EE + HPHN-triplet
- Best recall@1 of EE + HPHN-triplet: 0.745 (0.716 in paper)
- Note: The meaning of epoch in the paper and implementation is 'number of classes * number of instance per class'. So, the total epochs can be seen unusually high but it does not indicate slow convergence.
# EE + HPHN-triplet (n_inner_points=2)
$ python3 main.py --gpu_idx=0 --data_name=cars196 \
--backbone=googlenet --embed_dim=512 \
--loss=hphn_triplet --n_inner_pts=2 --ee_l2norm=True \
--seed=0 --batch_size=128 --image_size=227 \
--optim=adam --lr=0.0001 --lr_decay_factor=0.5 --lr_decay_epochs=400,800,1200,1600 \
--epochs=4000 --eval_epoch_term=5 --save_dir=results/inner-2
# HPHN-triplet (n_inner_points=0)
$ python3 main.py --gpu_idx=0 --data_name=cars196 \
--backbone=googlenet --embed_dim=512 \
--loss=hphn_triplet --n_inner_pts=0 --ee_l2norm=True \
--seed=0 --batch_size=128 --image_size=227 \
--optim=adam --lr=0.0001 --lr_decay_factor=0.5 --lr_decay_epochs=400,800,1200,1600 \
--epochs=4000 --eval_epoch_term=5 --save_dir=results/inner-0
$ tensorboard --logdir=results --port=10000
- dark blue: EE + HPHN-triplet (n_inner_points=2)
- sky blue: HPHN-triplet (n_inner_points=0)
If you find Embedding Expansion useful in your research, please consider to cite the following paper.
@inproceedings{ko2020embedding,
title={Embedding Expansion: Augmentation in Embedding Space for Deep Metric Learning},
author={Ko, Byungsoo and Gu, Geonmo},
booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
year={2020}
}
This repository is refactored based on the implementation of Combination of Multiple Global Descriptors for Image Retrieval (CGD). (Special thanks to the authors :D)
If you are interested in extended implementation of deep metric learning, please check the CGD's repository and consider to cite the following paper.
@article{jun2019combination,
title={Combination of Multiple Global Descriptors for Image Retrieval},
author={Jun, HeeJae and Ko, ByungSoo and Kim, Youngjoon and Kim, Insik and Kim, Jongtack},
journal={arXiv preprint arXiv:1903.10663},
year={2019}
}
Copyright (c) 2020-present NAVER Corp.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.