Skip to content

TGANet: Text-guided attention for improved polyp segmentation [Early Accepted & Student Travel Award at MICCAI 2022]

Notifications You must be signed in to change notification settings

nikhilroxtomar/TGANet

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

19 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

TGANet: Text-guided attention for improved polyp segmentation

1. Abstract

Colonoscopy is a gold standard procedure but is highly operator-dependent. Automated polyp segmentation, a precancerous precursor, can minimize missed rates and timely treatment of colon cancer at an early stage. Even though there are deep learning methods developed for this task, variability in polyp size can impact model training, thereby limiting it to the size attribute of the majority of samples in the training dataset that may provide sub-optimal results to differently sized polyps. In this work, we exploit size-related and polyp number-related features in the form of text attention during training. We introduce an auxiliary classification task to weight the text-based embedding that allows network to learn additional feature representations that can distinctly adapt to differently sized polyps and can adapt to cases with multiple polyps. Our experimental results demonstrate that these added text embeddings improve the overall performance of the model compared to state-of-the-art segmentation methods. We explore four different datasets and provide insights for size-specific improvements. Our proposed text-guided attention network (TGANet) can generalize well to variable-sized polyps in different datasets.

2. Architecture

3. Implementation

The proposed architecture is implemented using the PyTorch framework (1.9.0+cu111) with a single GeForce RTX 3090 GPU of 24 GB memory.

3.1 Dataset

We have used the following datasets:

All the dataset follows an 80:10:10 split for training, validation and testing, except for the Kvasir-SEG, where the dataset is split into training and testing.

3.2 Weight file

You can download the weight file from the following links:

3. Results

3.1 Quantative Results

3.2 Qualitative Results

4. Citation

@inproceedings{tomar2022tganet
title={TGANet: Text-guided attention for improved polyp segmentation},
author={Tomar, Nikhil Kumar and Jha, Debesh and Bagci, Ulas and Ali, Sharib},
booktitle={arXiv preprint arXiv:2205.04280},
year={2022}
} 

4. License

The source code is free for research and education use only. Any comercial use should receive a formal permission from the first author.

6. Contact

Please contact [email protected] for any further questions.

About

TGANet: Text-guided attention for improved polyp segmentation [Early Accepted & Student Travel Award at MICCAI 2022]

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages