Semantic Segmentation is done by assigning each pixel in the image a class. This is a key problem in th field of computer vision. In this repo, we would look at using UNet to do this:

Anaconda (download from here)

1. Clone the Repo
2. open a terminal (bash)
   

In bash type the following

cd UNet
conda env create -f=environment.yml # this would download the necessary python packages from anaconda cloud
conda activate my_unet
cd data
tar -xvzf iccv09Data.tar.gz
mv iccv09Data raw # renames folder to raw for data integrity
cd ..
python src/processing.py
python src/train_unet.py

type the following in your bash

jupyter notebook

After running src/train_unet.py, you should have some weights. You can run the follwing step to see your results.

python src/load_weights_and_test.py -w <PATH OF TRAINED MODEL>

The data is taken from ICCV09 [1] which contains 715 images sourced from LabelMe, MSRC, PASCAL VOC, and Geometric Context. Labels were obtained using Amazon's Mechanical Turk (AMT). Data can be downloaded from here

The labels are:

• horizons.txt : image dimensions and location of horizon
• labels/*.regions.txt : integer matrix indicating each pixel's semantic class (sky, tree, road, grass, water, building, mountain, or foreground object). A negative number indicates unknown.
• labels/*.surfaces.txt : integer matrix indicating each pixel's geometric class (sky, horizontal, or vertical).
• labels/*.layers.txt : integer matrix indicating distinct image regions.

UNet[2] is Convolutional Neural Network (CNN) which builds on the well known FCN[3]. Compared to the traditional sliding window approach, which won the Electron Microscopy Segmentation challenge at ISBI 2012, the FCN pose a more elegant architecture which reduces redundant overlapping patches its predecessor had.

The Unet consists of 23 convolutional layers with one contraction and one, more or less symmetric, expansion path. A concatenation of high resolution features from the contracting path to the unsampled features from the expanding path allows for localization.

The UNet architecture can be seen below

In this repo, we would be using a small version of UNet and train it over 50 epoch. The accuracy_loss_plot is shown below

Accuracy plot

Loss plot

[1] Decomposing a Scene into Geometric and Semantically Consistent Regions, Stephen Gould, Richard Fulton, Daphne Koller, ICCV 2009

[2] U-Net: Convolutional Networks for Biomedical Image Segmentation. Olaf Ronneberger, Philipp Fischer, Thomas Brox

[3] Fully Convolutional Networks for Semantic Segmentation. Jonathan Long, Evan Shelhamer, Trevor Darrell. UN Berkeley