Skip to content

🔒 Implementation of Shokri et al(2016) "Membership Inference Attacks against Machine Learning Models"

License

Notifications You must be signed in to change notification settings

yonsei-sslab/MIA

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

38 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

MIA_ML

Modifications were made on shadow models' training methodology in order to prevent overfitting

  1. Added weight decay factor
  2. Implemented early stopping
  3. Loads & saves best model based on evaluation metrics
  4. Creates member vs non-member attack dataset based on shadow testset

How to run

  1. (Optional) Customize train / inference configurations in config.yaml

  2. (Optional) python train_target.py: Train the victim model which is the target of the extraction.

  3. python train_shadow.py: Corresponds to Diagram 1-1 ~ Diagram 2-2 illustrated below.

  4. python train_attack.py: Corresponds to Diagram 2-3 ~ Diagram 3 illustrated below.

  5. python inference_attack.py: Corresponds to Diagram 4 illustrated below.

Result

  • Replicated the paper's configuration on config.yaml
  • ROC Curve is plotting TPR / FPR according to MIA classification thresholds
MIA Attack Metrics Accuracy Precision Recall F1 Score
CIFAR10 0.7761 0.7593 0.8071 0.7825
CIFAR100 0.9746 0.9627 0.9875 0.9749
MIA ROC Curve CIFAR10 MIA ROC Curve CIFAR100
roc_curve CIFAR10 roc_curve CIFAR100

Paper's Methodology in Diagrams

Page2

Page3

Page4

Page5

Page6

Page7

Page8

Page9

Page10

About

🔒 Implementation of Shokri et al(2016) "Membership Inference Attacks against Machine Learning Models"

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages