Skip to content

cogsys-tuebingen/prdarts

Repository files navigation

Prune and Replace NAS

by Kevin Alexander Laube and Andreas Zell, on arxiv

This code is based on the implementations of DARTS and P-DARTS.

Idea:

Iteratively pruning and replacing bad candidates from the operation pool enables us to efficiently search through vast operation spaces in reasonable time. We use DARTS to rank the currently available candidates, prune the worst ones, and generate new ones by applying network morphisms to those that are left.

Run:

Sample scripts that match our experiments in the paper are provided in the ./scripts folder.

Search

The progress and discovered cells of our DL2 search, these stats are plotted to tensorboard.

Search

DL2normal DL2reduction

Retraining

Test error (%) on CIFAR-10 and CIFAR-100 in comparison. The focus of this research is the candidate operation space, thus the only major change compared to the DARTS baseline is the progressive pruning and replacement schedule.

Method #params #ops GPU days CIFAR-10 CIFAR-100
NASNet-A 3.3M 13 1800 2.65
AmoebaNet-B 2.8M 19 3150 2.55
ENAS 4.6M 5 0.5 2.89
DARTS (1st order) 2.9M 8 1.5 2.94
DARTS (2nd order) 3.4M 8 4 2.83
P-DARTS C10 3.4M 8 0.3 2.5 16.55
P-DARTS C100 3.6M 8 0.3 2.62 15.92
sharpDARTS 3.6M 0.8 2.45
SNAS moderate 2.8M 8 1.5 2.85
NASP 3.3M 7 0.2 2.8
NASP (more ops) 7.4M 12 0.3 2.5
PR-DARTS DL1 3.2M 15/15 0.82 2.74 17.37
PR-DARTS DL2 4.0M 15/15 0.82 2.51 15.53
PR-DARTS DR 4.2M 26/39 0.88 2.55 16.69
PR-DARTS UR 5.4M 45/83 1.10 3.79

Citation

If you use any part of this code in your research, please cite our paper:

@article{laube2019prnas,
  title={Prune and Replace NAS},
  author={Kevin A. Laube and Andreas Zell},
  journal={arXiv preprint https://arxiv.org/abs/1906.07528},
  year={2019}
}

About

Code for our paper "Prune and Replace NAS"

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published