Sonnet: A self-guided ordinal regression neural network for segmentation and classification of nuclei in large-scale multi-tissue histology images
This repository contains a tensorflow implementation of SONNET: a self-guided ordinal regression neural network that performs simultaneously nuclei segmentation and classification. By introducing a distance decreasing discretization strategy, the network can detect nuclear pixels (inner pixels) and high uncertainty regions (outer pixels). The self-guided training strategy is applied to high uncertainty regions to improve the final outcome.
As part of this research, we introduce a new dataset for Gastric Lymphocyte Segmentation And Classification (GLySAC), which includes 59 H&E stained image tiles, of size 1000x1000. More details can be found in the SONNET paper.
The repository includes:
- Source code of SONNET.
- Training code for SONNET.
- Datasets employed in the SONNET paper.
- Pretrained weights for the SONNET encoder.
- Evaluation on nuclei segmentation metrics (DICE, AJI, DQ, SQ, PQ) and nuclei classification metrics (Fd and F1 scores).
conda create --name sonnet python=3.6
conda activate sonnet
pip install -r requirements.txt
Download the CoNSeP dataset from this link.
Download the PanNuke dataset from this link
Download the MoNuSAC and GLySAC from this link
To help with debugging and applying the model for nuclei segmentation and classification, it requires four steps:
To train the model, the data needs to be extracted into patches. First, set the dataset name in config.py. Then, To extract data into patches for training, simply run:
python extract_patches.py
The patches are numpy arrays with the shape of [RGB, inst, type], where RGB is the input image, inst is the foreground/background groundtruth, type is the type groundtruth map.
Download the pretrained-weights of the encoder used in SONNET on this link
Before training the network:
- Set the training dataset and validation dataset path in
config.py. - Set the pretrained-weights of the encoder in
config.py. - Set the log path in
config.py. - Change the hyperparameters used in training process according to your need in
opt/hyperconfig.py.
To train the network with GPUs 0 and 1:
python train.py --gpu='0,1'
Before testing the network on the test dataset:
- Set path to the test dataset, path to the model trained weights, path to save the output in
config.py. Achieve the network prediction by the command:
python infer.py --gpu='0'
It is notice that the inference only support for 1 GPU only.
To obtain the final outcome, i.e. the instance map and the type of each nuclear, run the command:
python process.py
To calculate the metrics used in SONNET paper, run the command:
- instance segmentation:
python compute_stats.py --mode=instance --pred_dir='pred_dir' --true_dir='true_dir' - type classification:
python compute_stats.py --mode=type --pred_dir='pred_dir' --true_dir='true_dir'
|
|
Type of each nucleus is represented by the color:
- Pink for Epithelial.
- Yellow for Lymphocyte.
- Blue for Miscellaneous.
Python 3.6, Tensorflow 1.12 and other common packages listed in requirements.txt
The datasets that we used in SONNET are from these papers:
If you use any of these dataset for your research, please have a citation of it.