Skip to content

[NeurIPS-W'21] Generative Optimization Nets for Memory-Efficient Data Generation

License

Notifications You must be signed in to change notification settings

imperial-qore/GON

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

95 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Generative Optimization Networks

GON for Memory-Efficient Data Generation

Hypothesis: "A discriminator is all you need". A sufficiently trained discriminator could not only indicate whether an input belongs to a data distribution but also how to tweak the input to make it resemble more closely to the target distribution. Thus, we can use only a discriminator for data generation and use those as fake samples in a self-adversarial training fashion. This allows us to reduce the parameter size significantly compared to traditional GANs. GON stands for Generative Optimization Network and is an ML framework that is similar to GANs but does not use a generator. GONs use a single discriminator network that also generated new data samples by maximizing discriminator output by modigying the input (i.e. neural network inversion). Without the generator, GONs enable significant gains in terms of memory footprints and allow us to deploy generative models in resource constrined Edge devices.

In this repo, we use a GON to train a reconstruction based time-series anomaly detector. Experiments on a Raspberry-Pi testbed with two existing and a new suite of datasets show that our framework gives up to 32% higher detection F1 scores and 58% lower memory consumption, with only 5% higher training overheads compared to the state-of-the-art.

Result Reproduction

Install dependencies:

python3 -m pip install -r requirements.txt
python3 -m pip install -r torch==1.9.1+cpu torchvision==0.10.1+cpu -f https://download.pytorch.org/whl/torch_stable.html

Run model <M> on dataset <D>:

python3 main.py --model <M> --dataset <D> --retrain

This will train the model and give results. <M> can be one of GON, USAD, MAD_GAN, SlimGAN, DILOF, IF, TranAD, ONLAD, SVM. <D> can be one of SMD, MSDS, FTSAD-1, FTSAD-25, FTSAD-55. All preprocessed datasets are in the processed folder.

To train all models at once:

./train.sh

To generate accuracy scores for model <M> on dataset <D>:

python3 main.py --model <M> --dataset <D> --test

To generate the memory consumpion profile of a model:

python3 main.py --model <M> --dataset <D> --memory

Gitpod

You can directly run tests and generate results using a Gitpod Workspace without needing to install anything on your local machine. Click "Open in Gitpod" on top of the page and test the code by running python3 main.py --model GON --dataset FTSAD-1 --retrain.

Supplementary video

IMAGE ALT TEXT HERE

External Links

Items Contents
Pre-print https://arxiv.org/pdf/2110.02912.pdf
Supplementary https://github.com/imperial-qore/GON_MNIST
Video https://youtu.be/NN79i7hB3-s
Contact Shreshth Tuli (@shreshthtuli)
Funding Imperial President's scholarship

Cite this work

Our work is published in NeurIPS 2021, Workshop on ML for Systems. Cite our work using the bibtex entry below.

@article{tuli2021generative,
  title={Generative Optimization Networks for Memory Efficient Data Generation},
  author={Tuli, Shreshth and Tuli, Shikhar and Casale, Giuliano and Jennings, Nicholas R},
  journal={Advances in Neural Information Processing Systems, Workshop on ML for Systems},
  year={2021}
}

License

BSD-3-Clause. Copyright (c) 2021, Shreshth Tuli. All rights reserved.

See License file for more details.