In this project, we explore how KNN improves the reliability of neural networks, including two parts:

• how model performs on data samples with different KNN distances
• whether the KNN distances of out-of-distribution dataset exhibits different distribution compared with in-distribution

Additionally, we provide a demo for Conformal Prediction.

This project is motivated by the following two works:

@article{sun2022knnood,
  title={Out-of-distribution Detection with Deep Nearest Neighbors},
  author={Sun, Yiyou and Ming, Yifei and Zhu, Xiaojin and Li, Yixuan},
  journal={ICML},
  year={2022}
}

@article{yuksekgonul2023beyond,
  title={Beyond Confidence: Reliable Models Should Also Consider Atypicality},
  author={Yuksekgonul, Mert and Zhang, Linjun and Zou, James and Guestrin, Carlos},
  journal={arXiv preprint arXiv:2305.18262},
  year={2023}
}

We use CIFAR10 and CIFAR100 as in-distribution datasets on which we conduct experiments to answer the first problem. We solve the second problem on a common out-of-distribution dataset: SVHN.

bash scripts/cifar10.sh 

bash scripts/cifar100.sh 

bash scripts/svhn.sh