The repository of the project Effects of Adversarial Attacks on Time-Series Forecasting Tasks by Göktuğ Öcal for CMPE588 Testing and Verification in Machine Learning course in Boğaziçi University.

Since the decision-making process in some businesses highly relies on Deep Neural Networks and there are lots of intelligent systems that use DNNs, the robustness of the DNNs is becoming more and more important every day in nearly every domain. Recent studies have shown that DNNs are vulnerable to perturbated attacks called adversarial attacks, which makes them non-robust. In the research community, the main example of the importance of the robustness of DNNs is the autonomous cars and car crashes. Yet there are other applications of DNNs such as time series forecasting which is very crucial for decision-making in businesses. In this study, I have investigated the effects of adversarial attacks on different DNN models, mostly based on Long Short-Term Memory(LSTM) networks and CNN based forecasting models. The project has couple of results including those topics;

• Adversarial Attacks to Forecasting Models.
• Adversarial Training for Improving the Robustness of Models

1- Install required packages.

$ pip install -r requirements.txt

Trains the selected models for selected datasets and saves models and error metrics to a specific directory.

$ python model_training.py [--setting <"mtm | mto">] [--datasets <"dataset_name_1, dataset_name_2, ...">] [--models <"model_name_1, model_name_2, ...">]

Attacks to the models in a specific directory where the model training step saves at.

$ python attacking_to_models.py [--setting <"mto, mtm">] [--attack <"FGSM, PGD">]

Trains the models after generating adversarial samples that is added to training and testing tests. Saves the trained models with their error metrics.

$ python adversarial_training.py [--datasets <"dataset_name_1, dataset_name_2, ...">] [--models <"model_name_1, model_name_2, ...">] [--attack <"FGSM, PGD">]

Model Name	Structure
Single Layer LSTM
Double Layer LSTM
Bidirectional LSTM
CNN1D

(Many-to-one) Original sample, perturbation, and Adversarial sample generated with PGD attack and LSTM model for Solar Generation dataset with input size of 23:

Original Sample	Perturbation by PGD	Adversarial Sample

(Many-to-one) Original sample, perturbation, and Adversarial sample generated with PGD attack and LSTM model for Electricity Transformer dataset with input size of 92:

Original Sample	Perturbation by PGD	Adversarial Sample

(Many-to-one) Original sample, perturbation, and Adversarial sample generated with PGD attack and CNN model for Electricity Transformer dataset with input size of 92:

Original Sample	Perturbation by PGD	Adversarial Sample

(Many-to-many) Original sample, perturbation, and Adversarial sample generated with PGD attack and LSTM model for Electricity Transformer dataset with input size of 168:

Original Sample	Perturbation by PGD	Adversarial Sample

(Many-to-one) Comparison of Adversarial Samples generated with FGSM and PGD attacks and LSTM model for Solar Generation dataset with input size of 23

(Many-to-one) Comparison of Adversarial Samples generated with FGSM and PGD attacks and LSTM model for Electricity Transformer dataset with input size of 92

(Many-to-one) Comparison of Adversarial Samples generated with FGSM and PGD attacks and CNN model for Electricity Transformer dataset with input size of 92:

(Many-to-many) Comparison of Adversarial Samples generated with FGSM and PGD attacks and LSTM model for Electricity Transformer dataset with input size of 168:

All the results are the Mean Absolute Errors (MAE) values. Original column refers to forecasting error for the model trained with original data. Attack columns refers to forecasting error for adversarial samples generated with corresponding attacks type and given to the model trained with original data. Training and Test columns are the forecasting errors for the models trained with both original and adversarial samples and both training and test set includes both of the samples.