This is an implementation of Mask R-CNN on Python 3, Keras, and TensorFlow. The model generates bounding boxes and segmentation masks for each instance of an object in the image. It's based on Feature Pyramid Network (FPN) and a ResNet101 backbone.

The repository includes:

• Source code of Mask R-CNN built on FPN and ResNet101.
• Training code for MS COCO
• Pre-trained weights for MS COCO
• Jupyter notebooks to visualize the detection pipeline at every step
• ParallelModel class for multi-GPU training
• Evaluation on MS COCO metrics (AP)
• Example of training on your own dataset

The code is documented and designed to be easy to extend. If you use it in your research, please consider citing this repository (bibtex below). If you work on 3D vision, you might find our recently released Matterport3D dataset useful as well. This dataset was created from 3D-reconstructed spaces captured by our customers who agreed to make them publicly available for academic use. You can see more examples here.

• demo.ipynb Is the easiest way to start. It shows an example of using a model pre-trained on MS COCO to segment objects in your own images. It includes code to run object detection and instance segmentation on arbitrary images.

• train_shapes.ipynb shows how to train Mask R-CNN on your own dataset. This notebook introduces a toy dataset (Shapes) to demonstrate training on a new dataset.

• (model.py, utils.py, config.py): These files contain the main Mask RCNN implementation.

• inspect_data.ipynb. This notebook visualizes the different pre-processing steps to prepare the training data.

• inspect_model.ipynb This notebook goes in depth into the steps performed to detect and segment objects. It provides visualizations of every step of the pipeline.

• inspect_weights.ipynb This notebooks inspects the weights of a trained model and looks for anomalies and odd patterns.

To help with debugging and understanding the model, there are 3 notebooks (inspect_data.ipynb, inspect_model.ipynb, inspect_weights.ipynb) that provide a lot of visualizations and allow running the model step by step to inspect the output at each point. Here are a few examples: