Code for the paper Robustness and Accuracy Could Be Reconcilable by (Proper) Definition (ICML 2022).

This project is tested under the following environment settings:

• OS: Ubuntu 20.04.3
• GPU: NVIDIA A100
• Cuda: 11.1, Cudnn: v8.2
• Python: 3.9.5
• PyTorch: 1.8.0
• Torchvision: 0.9.0

The codes are modifed based on the PyTorch implementation of Rebuffi et al., 2021.

• Install or download AutoAttack:

pip install git+https://github.com/fra31/auto-attack

• Download 1M DDPM generated data from the official implementation of Rebuffi et al., 2021:

To run the KL-based baselines (with 1M DDPM generated data), an example is:

python train-wa.py --data-dir 'cifar-data' \
    --log-dir 'trained_models' \
    --desc 'WRN28-10Swish_cifar10s_lr0p2_TRADES5_epoch400_bs512_fraction0p7_ls0p1' \
    --data cifar10s \
    --batch-size 512 \
    --model wrn-28-10-swish \
    --num-adv-epochs 400 \
    --lr 0.2 \
    --beta 5.0 \
    --unsup-fraction 0.7 \
    --aux-data-filename 'cifar10_ddpm.npz' \
    --ls 0.1

Here --ls 0.1 is inherent from the the code implementation of Rebuffi et al., 2021.

To run our methods (with 1M DDPM generated data), an example is:

python train-wa.py --data-dir 'cifar-data' \
    --log-dir 'trained_models' \
    --desc 'WRN28-10Swish_cifar10s_lr0p2_TRADES4_epoch400_bs512_fraction0p7_LSE' \
    --data cifar10s \
    --batch-size 512 \
    --model wrn-28-10-swish \
    --num-adv-epochs 400 \
    --lr 0.2 \
    --beta 4.0 \
    --unsup-fraction 0.7 \
    --aux-data-filename 'cifar10_ddpm.npz' \
    --LSE --ls 0

Here we only need to activate the flag --LSE and set --ls 0.

Below are pretrained checkpoints of WRN-28-10 (Swish) and WRN-70-16 (Swish) with --beta=3.0:

• Downloading checkpoint to trained_models/mymodel/weights-best.pt
• Downloading argtxt to trained_models/mymodel/args.txt

For evaluation under AutoAttack, run the command (taking our method as an example):

python eval-aa.py --data-dir 'cifar-data' \
    --log-dir 'trained_models' \
    --desc mymodel

To re-implement the toy demos, we could run:

python generate_toy_demo.py --demo PGDAT --divergence KL --divergence_C L1

The flag --demo refers to the objective used for training; --divergence refers to the used metric loss; --divergence_C refers to the metric calculating SCORE values. The Figures 1 and 2 in our paper are plotted by MATLAB using the script plot_toy_demo.m.

If you find the code useful for your research, please consider citing

@inproceedings{pang2022robustness,
  title={Robustness and Accuracy Could be Reconcilable by (Proper) Definition},
  author={Pang, Tianyu and Lin, Min and Yang, Xiao and Zhu, Jun and Yan, Shuicheng},
  booktitle={International Conference on Machine Learning (ICML)},
  year={2022}
}

and/or our related works

@inproceedings{wang2023better,
  title={Better Diffusion Models Further Improve Adversarial Training},
  author={Wang, Zekai and Pang, Tianyu and Du, Chao and Lin, Min and Liu, Weiwei and Yan, Shuicheng},
  booktitle={International Conference on Machine Learning (ICML)},
  year={2023}
}

@inproceedings{pang2021bag,
  title={Bag of Tricks for Adversarial Training},
  author={Pang, Tianyu and Yang, Xiao and Dong, Yinpeng and Su, Hang and Zhu, Jun},
  booktitle={International Conference on Learning Representations (ICLR)},
  year={2021}
}