MEAD is a framework for evaluating detectors based on several attack strategies.
The files with the adversarial samples and the features have been created by executing this code.
Package
MEAD/
├── adv_data
│ ├── cifar10
│ └── mnist
├── checkpoints
│ ├── rn-best.pt
│ └── small_cnn-best.pt
├── features
│ ├── fs
│ │ ├── cifar10
│ │ │ └── evaluation
│ │ └── mnist
│ │ └── evaluation
│ ├── kd_bu
│ │ ├── cifar10
│ │ │ └── evaluation
│ │ └── mnist
│ │ └── evaluation
│ ├── lid
│ │ ├── cifar10
│ │ │ └── evaluation
│ │ └── mnist
│ │ └── evaluation
│ ├── magnet
│ │ ├── cifar10
│ │ │ └── evaluation
│ │ └── mnist
│ │ └── evaluation
│ └── nss
│ ├── cifar10
│ │ └── evaluation
│ └── mnist
│ └── evaluation
├── __init__.py
├── mead_evaluation.py
├── mead.yml
├── models
│ ├── __init__.py
│ ├── resnet.py
│ └── small_cnn.py
├── README.md
├── results
├── setup_paths.py
└── utils
├── general_utils.py
├── __init__.py
└── plot_utils.py
To execute MEAD:
- Create the enviroment for MEAD:
foo@bar:~$ conda create --name mead python=3.8- Activate the enviroment for MEAD:
foo@bar:~$ source activate mead- Install all the required packages:
(mead) foo@bar:~$ pip3 install -r requirements.txt- Launch the test from CLI for CIFAR10:
-d: dataset between cifar10 and mnist
-m: detector to evaluate between fs, kd_bu, lid, magnet and nss
-p: plot name to save the ROC
-dev: device between cuda and cpu
(mead) foo@bar:~$ python mead_evaluation.py -d cifar10 -m kd_bu -p cifar10_kd_bu -dev cuda We run each experiment on a machine equipped with an Intel(R) Xeon(R) CPU E5-2623 v4, 2.60GHz clock frequency, and a GeForce GTX 1080 Ti GPU.