This AI project is a detection task for identifying zebrafish organs and phenotypes in micrographs, which is based on the Meta AI project, Detectron2 version 0.4.1. It mainly used Mask R-CNN for training and validating. It has 16 detected objects, including 8 specific organs and 8 specific abnormal phenotypes.

Inference results by Mask R-CNN

To use our zebrafish AI detection, you have to learn how to install and utilize the detectron2, please refer the detectron2 instructions.

Firstly, install python3.8 or higher version, detectron2 (0.4.1) and other dependent libraries (see zebrafish_maskrcnn.py). Among all the libraries, fishutil and fishclass are two customized python libraries for transforming the model outputs to quantitative parameters of zebrafish.

Then put the pre-trained model weights in your specified path.

mask r-cnn & resnet_r_101/tensormask & resnet_r_50: download here

test_image:/train_image: download here test train

# Modify some necessary paths
# path to model weights
cfg.MODEL.WEIGHTS = os.path.join(cfg.OUTPUT_DIR, "model_final.pth")
...
# path to images that pending to detect
path = 'your/specified/path'
...
# save your results
df.to_csv(os.path.join('.../results/', 'results.csv'))

Finally, run the inference.py file in inference mode locally.

# get help
# specify the detect images and image type as well as the output folder
python inference.py --help

If you want to use TensorMask to identify the objects, please refer to another python file, zebrafish_tensormask.py. Through this model you can also see the identification results in real-time, which is supported by Open CV.

We have opened our image library and annotations, if you want to re-train this model locally, please download the train.json, test.json, train images and test images(see images_url.txt). The json file contains annotations of COCO style. After downloading the images and files, register the coco instances in your code, modify the name and path of the register.

# train registry
from detectron2.data.datasets import register_coco_instances
register_coco_instances("zebrafish_train", {}, "/your/path/train.json",
                        "/your/path/train_images/")
register_coco_instances("zebrafish_train", {}, "/your/path/test.json",
                        "/your/path/test_images/")

After the model inference, we can acquire a csv file which cotains the quantitative parameters of specific organs and abnormal phenotypes, these information demonstrate every detail of one specific zebrafish, and we can use them to analyze the developmental status of each zebrafish.

We will continually update this repository and add more images to our library, if you want to use this work, please cite our research article. Citation infos:{ Deep Learning-Enabled Morphometric Analysis for Toxicity Screening Using Zebrafish Larvae Gongqing Dong, Nan Wang, Ting Xu, Jingyu Liang, Ruxia Qiao, Daqiang Yin, and Sijie Lin Environmental Science & Technology 2023 57 (46), 18127-18138 DOI: 10.1021/acs.est.3c00593 }