A library for distribution-free model-agnostic finite-sample uncertainty quantification applied to computer vision tasks.!

A library for Conformal Classification, Object Detection and Segmentation, based on PyTorch, with all the latest methods and models!

Conformal prediction is a framework in machine learning that provides a way to quantify the uncertainty of predictions. It goes beyond traditional point predictions by providing a set of possible outcomes along with confidence levels. This allows for more reliable and interpretable predictions, especially in situations where the training data is limited or noisy. Conformal prediction can be applied to various tasks, including classification, regression, object detection, and segmentation.

CODS is a vision-oriented python library that implements conformal prediction for classification, object detection and (soon) segmentation. It is built on top of PyTorch for efficiency (with CUDA support), and is designed to allow for easy addition of new conformal scores, procedures, models or dataset, and for quick benchmarking of those.

pip install -e git+https://github.com/leoandeol/cods.git

git clone https://github.com/leoandeol/cods.git
cd cods
pip install -r requirements.txt
pip install -e .

Application of CODS to the image of a kitchen in MS-COCO with $α=0.1$

Introduction to Conformal Classification

Introduction to Conformal Object Detection

[Work in Progress]

1. Classification: [Implemented]
2. Regression: [Not Implemented]
3. Object Detection: [Implemented]
4. Segmentation: [Work in Progress]

This project was developped by Léo Andéol & Luca Mossina as part of the DEEL project.

1. Vovk et al. 2005: Algorithmic Learning in a Random World
2. Angelopoulos et al. 2021: Conformal Prediction: A Gentle Introduction
3. Shafer et al. 2008: A Tutorial on Conformal Prediction
4. Angelopoulos et al. 2022: Conformal Risk Control
5. Park et al. 2020: PAC Confidence Sets for Deep Neural Networks via Calibrated Prediction
6. Bates et al 2019: Risk Controlling Prediction Sets
7. Andéol et al. 2023: Confident Object Detection via Conformal Prediction and Conformal Risk Control
8. Li et al. 2022: Towards PAC Multi-Object Detection and Tracking
9. Bates et al. 2021: Risk Controlling Prediction Sets

If you use CODS in your research, please cite the following reference:

@misc{andeol2024cods, 
    title={Conformal Object Detection \& Segmentation},
    author={Andéol, Léo and Mossina, Luca},
    year={2024},
    howpublished={https://github.com/leoandeol/cods},
}

The package is released under the MIT License.