This is the official implementation of Mining Negative Temporal Contexts For False Positive Suppression In Real-Time Ultrasound Lesion Detection. We provide a real-time breast ultrasound detection solution to reduce about 50% FPs at a recall rate of 0.90. Also, we provide high-quality bounding box labels for the CVA-Net breast ultrasound video dataset. Our paper has been early accepted by MICCAI 2023!

[Arxiv] [PDF] [Code]

During ultrasonic scanning processes, real-time lesion detection can assist radiologists in accurate cancer diagnosis. However, this essential task remains challenging and underexplored. General-purpose real-time object detection models can mistakenly report obvious false positives (FPs) when applied to ultrasound videos, potentially misleading junior radiologists. One key issue is their failure to utilize negative symptoms in previous frames, denoted as negative temporal contexts (NTC). To address this issue, we propose to extract contexts from previous frames, including NTC, with the guidance of inverse optical flow. By aggregating extracted contexts, we endow the model with the ability to suppress FPs by leveraging NTC. We call the resulting model UltraDet. The proposed UltraDet demonstrates significant improvement over previous state-of-the-arts and achieves real-time inference speed.

The NTCA module leverages RoI-level NTC which are crucial for radiologists but ignored in previous works, thereby effectively improving the detection performance in a reliable and interpretable way. We plug the NTCA module into a basic real-time detector to form UltraDet. Experiments on CVA-BUS dataset demonstrate that UltraDet, with real-time inference speed, significantly outperforms previous works, reducing about 50% FPs at a recall rate of 0.90.

If you find UltraDet useful in your research, please cite our paper:

@article{yu2023mining,
  title={Mining Negative Temporal Contexts For False Positive Suppression In Real-Time Ultrasound Lesion Detection},
  author={Yu, Haojun and Li, Youcheng and Wu, QuanLin and Zhao, Ziwei and Chen, Dengbo and Wang, Dong and Wang, Liwei},
  journal={arXiv preprint arXiv:2305.18060},
  year={2023}
}

The whole project is based on detectron2.

# install anaconda3 with python3.9
conda create -n <env_name> python=3.9
conda activate <env_name>

# use tuna source for pypi (optional)
python -m pip install --upgrade pip

# install packages
conda install pytorch==1.10.1 torchvision==0.11.2 torchaudio==0.10.1 cudatoolkit=11.3 -c pytorch -c conda-forge
python -m pip install detectron2 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cu113/torch1.10/index.html
pip install pandas
pip install path
pip install sqlmodel
pip install coloredlogs
pip install opencv-p
pip install mmcv
pip install prettytable
pip install fire
pip install scipy
pip install timm

# install ultrasound_vid
git clone https://github.com/HaojunYu1998/UltraDet.git
cd UltraDet
pip install -e .

# issue shooting
# AttributeError: module 'distutils' has no attribute 'version'
pip uninstall setuptools
pip install setuptools==58.0.4

We are pleased to share the weights of the models required for UltraDet inference. You can access them through the following links:

After downloading the weights, please store them in the following path:

UltraDet/
  pretrained_models/
    flownet.ckpt
    r34.pkl
    ultradet.pth

We provide high quality annotation files. You can access them through the following links:

Please download PNG images from CVA-Net and arrange the images and labels in the COCO format. If you encounter any data-related issues, please contact us via [email protected].

Note: The authors of CVA-Net dataset have made corrections to the benign/malignant classification annotations (please see CVA-Net). To use our high-quality annotations, please first correct the video and image location in rawframes/.

UltraDet/
  datasets/
    bus_data_cva/
      rawframes/
        benign/
          2cda21c3aab26332/
            000000.png
            000001.png
            ...
          ...
        malignant/
          1dc9ca2f1748c2ec/
            000000.png
            000001.png
            ...
          ...
      test.json
      trainval.json

You can train or infer with UltraDet by running the following code:

# Training
python train_net.py --num-gpus {GPU_NUM} \
--config-file {CONFIG_FILES}

# Validation
python train_net.py --num-gpus {GPU_NUM} --eval-only --resume \
--config-file {CONFIG_FILES}

# Test
python train_net.py --num-gpus {GPU_NUM} --eval-only  --resume \
--config-file {CONFIG_FILES} \
DATASETS.SPLIT "test" \
DATASETS.NUM_FOLDS 1 \
# MODEL.WEIGHTS 'pretrained_models/ultradet.pth' # If you wish to load the pre-trained weights of UltraDet that we have released, please include this line of code