Multiple Instance Learning (MIL) methods are mainstream approaches for pathological image classification and analysis. The CAMELYON-16/17 datasets are commonly used to evaluate MIL methods. However, they have the following issues:

• CAMELYON-16/17 datasets contain some problematic slides
• Pixel-annotations of CAMELYON-16/17 test-dataset not accurate enough
• Different MIL methods do not have a unified dataset-split and evaluation-metrics on the CAMELYON dataset
• To conclude,there is no BENCHMARK for MIL methods

We do the following work to establish a CAMELYON-BENCHMARK

• Remove some problematic slides.
• Correct problematic annotations.
• Merge the CAMELYON-16/17 datasets and add some new slides to organize a larger, more balanced CAMELYON-NEW dataset.
• Evaluate mainstream MIL methods on the CAMELYON-NEW dataset.
• Evaluate mainstream feature extractors on the CAMELYON-NEW dataset.
• Use more comprehensive evaluation metrics to assess different methods.
• In summary, we establish a new CAMELYON-BENCHMARK.

• We apply balanced-dataset-split
• Details will be released soon

• BAIDU-PAN-LINK
• GOOGLE-PAN-LINK

• MEAN_MIL Attention-based Deep Multiple Instance Learning (ICML 2018)
• MAX_MIL Attention-based Deep Multiple Instance Learning (ICML 2018)
• AB_MIL Attention-based Deep Multiple Instance Learning (ICML 2018)
• TRANS_MIL Transformer based Correlated Multiple Instance Learning for WSI Classification (NeurIPS 2021)
• DS_MIL Dual-stream MIL Network for WSI Classification with Self-supervised Contrastive Learning (CVPR 2021)
• CLAM_MIL Data Efficient and Weakly Supervised Computational Pathology on WSI (NAT BIOMED ENG 2021)
• DTFD_MIL Double-Tier Feature Distillation MIL for Histopathology WSI Classification (CVPR 2022)
• RRT_MIL Towards Foundation Model-Level Performance in Computational Pathology (CVPR 2024)
• WIKG_MIL Dynamic Graph Representation with Knowledge-aware Attention for WSI Analysis (CVPR 2024)
• UPDATING...

• VIT_S (IMAGENT-PRETRAINED) Transformers for Image Recognition at Scale (ICLR 2021)
• PLIP (WSI-Contrastive-Learning) A visual–language model for WSI using medical Twitter (NAT MED 2023)
• UNI (WSI-PRETRAINED) Towards a general-purpose foundation model for computational pathology (NAT MED 2024)
• UPDATING...

• Obtain the patches on 20X Magnification
• Keep the Hyperparameter settings of original implement
• Use uniform,balanced dataset-split