This is a reference implementation of a Structure-from-Motion pipeline in OpenCV, following the work of Snavely et al. [2] and Hartley and Zisserman [1].
Note: This is not a complete and robust SfM pipeline implementation. The purpose of this code is to serve as a tutorial and reference for OpenCV users and a soft intro to SfM in OpenCV. If you are looking for a more complete solution with many more options and parameters to tweak, check out the following:
- OpenMVG https://openmvg.readthedocs.io/en/latest/#
- libMV https://developer.blender.org/tag/libmv/
- VisualSFM https://ccwu.me/vsfm/
- Bundler https://www.cs.cornell.edu/~snavely/bundler/
SfM-Toy-Library is now using OpenCV 3, which introduced many new convenience functions to Structure from Motion (see my blog post for details), making the implementation much cleaner and simpler.
Ceres solver was chosen to do bundle adjustment, for its simple API, straightforward modeling of the problem and long-term support.
Doxygen-style documentation comments appear throughout.
To compile use CMake: https://www.cmake.org
- OpenCV 3.x: https://www.opencv.org
- Ceres Solver (for bundle adjustment): https://ceres-solver.org/
- Boost C++ libraries v1.54+: https://www.boost.org/
Get Boost and Ceres using homebrew: brew install boost ceres-solver (you will need to tap homebrew/science for Ceres)
mkdir build
cd build
cmake -G "Xcode" ..
open SfMToyExample.xcodeproj
Obtain Boost (with e.g. apt-get install libboost-all-dev) and Ceres (probably need to clone and compile), or on OSX view homebrew as mentioned before.
mkdir build
cd build
cmake -G "Unix Makefiles" ..
make
Use Cmake's GUI to create a MSVC solution, and build it.
USAGE ./build/SfMToyUI [options] <input-directory>
-h [ --help ] produce help message
-d [ --console-debug ] arg (=2) Debug output to console log level (0 = Trace,
4 = Error).
-v [ --visual-debug ] arg (=3) Visual debug output to screen log level (0 =
All, 4 = None).
-s [ --downscale ] arg (=1) Downscale factor for input images
-p [ --input-directory ] arg Directory to find input images
Here's a place with some standard datasets for SfM: https://cvlabwww.epfl.ch/data/multiview/denseMVS.html
Also, you can use the "Crazy Horse" (A national memorial site in South Dakota) dataset, that I pictured myself, included in the repo.
Some relevant blog posts from over the years:
- https://www.morethantechnical.com/2016/10/17/structure-from-motion-toy-lib-upgrades-to-opencv-3/
- https://www.morethantechnical.com/2015/03/16/bootstrapping-planar-ar-tracking-without-markers-wcode/
- https://www.morethantechnical.com/2013/11/04/moving-to-qt-on-the-sfm-toy-library-project/
- https://www.morethantechnical.com/2012/08/09/checking-for-co-planarity-of-3d-points-in-opencv-wcode/
- https://www.morethantechnical.com/2012/08/09/decomposing-the-essential-matrix-using-horn-and-eigen-wcode/
- https://www.morethantechnical.com/2012/02/07/structure-from-motion-and-3d-reconstruction-on-the-easy-in-opencv-2-3-w-code/
- https://www.morethantechnical.com/2012/01/04/simple-triangulation-with-opencv-from-harley-zisserman-w-code/
- Multiple View Geometry in Computer Vision, Hartley, R. I. and Zisserman, A., 2004, Cambridge University Press [https://www.robots.ox.ac.uk/~vgg/hzbook/]
- Modeling the World from Internet Photo Collections, N. Snavely, S. M. Seitz, R. Szeliski, IJCV 2007 [https://phototour.cs.washington.edu/ModelingTheWorld_ijcv07.pdf]
- Triangulation, R.I. Hartley, P. Strum, 1997, Computer vision and image understanding [https://perception.inrialpes.fr/Publications/1997/HS97/HartleySturm-cviu97.pdf]
- Recovering baseline and orientation from essential matrix, B.K.P. Horn, 1990, J. Optical Society of America [https://people.csail.mit.edu/bkph/articles/Essential_Old.pdf]
- On benchmarking camera calibration and multi-view stereo for high resolution imagery. Strecha, Christoph, et al. IEEE Computer Vision and Pattern Recognition (CVPR) 2008. [https://infoscience.epfl.ch/record/126393/files/strecha_cvpr_2008.pdf]