This project provides Pytorch implementation for Angle-based Search Space Shrinking for Neural Architecture Search.
- Pytorch 1.3
- Python 3.5+
- Apex
The requirements.txt file lists other Python libraries that this project depends on, and they will be installed using: pip3 install -r requirements.txt
| Model | Flops | Top1 Acc. | Flops (ABS) | Top1 (ABS) | GoogleDrive |
|---|---|---|---|---|---|
| SPOS | 465M | 75.33% | 472M | 75.97% | Model |
| FairNAS | 322M | 74.24% | 325M | 74.42% | Model |
| ProxylessNAS | 467M | 75.56% | 470M | 76.14% | Model |
| DARTS | 530M | 74.88% | 619M (547M) | 75.59% (75.19%) | Model, Scale Down |
| PDARTS | 553M | 75.58% | 645M (570M) | 75.89% (75.64%) | Model, Scale Down |
For the form x(y), x means models searched without human intervention, y means the models whose channels are scaled down to fit with the constraint of flops
We have splitted 50000 images from ImageNet Train Dataset as the validation set for search. The remainings are used for supernet training.
Run utils/get_flops_lookup_table.sh to generate flops lookup table which is required in Uniform Sampling.
Shrink search spaces with the following command:
cd shrinking
python3 -m torch.distributed.launch --nproc_per_node=8 main.py \
--train_dir YOUR_TRAINDATASET_PATH
Note: SPOS and ProxylessNAS share the same shrunk search space. DARTS and PDARTS share the same shrunk search space
cd darts-master
python3 train_search.py --data $YOUR_DATA_PATH --unrolled --save DARTS_ABS \
--operations_path $YOUR_SHRUNK_SEARCH_SPACE
cd pdarts-master
python3 train_search.py --save PDARTS_ABS --tmp_data_dir $YOUR_DATA_PATH \
--operations_path $YOUR_SHRUNK_SEARCH_SPACE
cd searching
python3 imagenet_arch_search.py --path ABS
--target_hardware flops \
--operations_path $YOUR_SHRUNK_SEARCH_SPACE \
--train_dir $YOUR_TRAINDATASET_PATH --test_dir $YOUR_TESTDATASET_PATH
tmux new -s mq_server
sudo apt update
sudo apt install rabbitmq-server
sudo service rabbitmq-server start
sudo rabbitmqctl add_user test test
sudo rabbitmqctl set_permissions -p / test '.*' '.*' '.*'
Before search, please modify host and username in the config file searching/config.py.
cd searching
python3 -m torch.distributed.launch --nproc_per_node=8 main.py --operations_path \
--train_dir $YOUR_TRAINDATASET_PATH --test_dir $YOUR_TESTDATASET_PATH
$YOUR_SHRUNK_SEARCH_SPACE
tmux new -s server_x
cd searching
python3 test_server.py
You can start more than one test_server.py to speed up, if you have enough GPUs and memory researces.
Finally, train and evaluate the searched architectures with the following command.
Train:
cd training
python3 -m torch.distributed.launch --nproc_per_node=8 train_from_scratch.py \
--train_dir $YOUR_TRAINDATASET_PATH --test_dir $YOUR_TESTDATASET_PATH
Evaluate:
cd training
python3 -m torch.distributed.launch --nproc_per_node=8 train_from_scratch.py \
--eval --eval-resume $YOUR_WEIGHT_PATH \
--train_dir $YOUR_TRAINDATASET_PATH --test_dir $YOUR_TESTDATASET_PATH
This implementation of ABS is based on DARTS, PDARTS, ProxylessNAS, SPOS, NAS-Benchmark-201. Except to replace the search space, everything else follows their original implementation. Please ref to their reposity for more details.
If you find that this project helps your research, please consider citing some of the following papers:
@article{hu2020abs,
title={Angle-based Search Space Shrinking for Neural Architecture Search},
author={Yiming Hu, Yuding Liang, Zichao Guo, Ruosi Wan, Xiangyu Zhang, Yichen Wei, \
Qingyi Gu, Jian Sun},
year={2020},
booktitle = {arXiv preprint arXiv:2004.13431},
}
@article{guo2019single,
title={Single path one-shot neural architecture search with uniform sampling},
author={Guo, Zichao and Zhang, Xiangyu and Mu, Haoyuan and Heng, Wen and Liu, \
Zechun and Wei, Yichen and Sun, Jian},
journal={arXiv preprint arXiv:1904.00420},
year={2019}
}