In this project, I have implemented the concept of Stereo Vision on three data sets, each of them containing 2 images of the same scenario but taken from two different camera angles. By comparing the information about the scene from 2 vantage points, we can obtain the 3D information by examining the relative position of the objects. I have also estimated the camera pose using the concept of the Essential matrix (E), Fundamental matrix (F) and, triangulation.
- Compare the two images and select the set of matching feature. Tune the Lowe's ration to reject the outliers
- Estimate the Fundamental matrix using the obtained matching feature. Use the RANSAC to make your estimation more robust. Enforce the rank 2 condition for the fundamental matrix.
- Estimate the Essential matrix(E) from the Fundamental matrix(F) and instrinsic camera parameter.
- Decompose the E into a translation T and rotation R
- Disambugiate the T and R using triangulation.
- Apply perspective transfomation to make sure that the epipolar lines are horizontal for both the images. This will limit the search space to horizontal line during the corrospondace matching process in the later stage.
- For each epipolar line, apply the sliding window with SSD to find the corrospondence and calulate disparity.
- Rescale the disparity to be from 0-255 for visualization
- Using the disparity calculated above, compute the depth map. The resultant image has a depth image instead of disparity.
-
Matching features in left and right image
-
Epipolar line corrosponding to the obtained matching feature
-
Rectified epipolar lines
-
Disparity and Depth map
-
Disparity and Depth heat map
- Python 2.0 or above
- OpenCV
Copyright (c) Jan 2023 Pradip Kathiriya