We release mpegflow for easy extraction of MPEG-flow (motion vectors) from video files along with a visualization tool vis, both under the MIT license. We provide Makefiles for Linux / Windows and distribute binary releases.

Please submit bugs on GitHub directly. For any other question, please contact Vadim Kantorov at [email protected] or [email protected].

If you use this code, please cite our work:

@inproceedings{kantorov2014,
      author = {Kantorov, V. and Laptev, I.},
      title = {Efficient feature extraction, encoding and classification for action recognition},
      booktitle = {Proc. Computer Vision and Pattern Recognition (CVPR), IEEE, 2014},
      year = {2014} }

Below is a visualization of some flow maps of MPI Sintel's alley_1 sequence:

Source	3	15	50
mpegflow
Ground truth

The tool accepts a video file path as command-line argument and writes MPEG-flow (motion vectors) to standard output. By default, the tool outputs the motion vectors arranged in two matrices - dx and dy.

The tool accepts a video file path and a dump directory as command-line arguments, output of mpegflow on standard input and saveson disk the visualization of motion vectors overlayed on video frames.

• To extract motion vectors:

  ./mpegflow examples/mpi_sintel_final_alley_1.avi > examples/alley_1.txt

• To visualize motion vectors:

  mkdir -p examples/vis_dump && ./mpegflow examples/mpi_sintel_final_alley_1.avi | ./vis examples/mpi_sintel_final_alley_1.avi examples/vis_dump

Feel free to copy-paste and run the examples above. More runnable examples are in examples/extract_motion_vectors.sh and examples/vis_motion_vectors.sh. Feel free to use vis.cpp and examples/vis_hue.m as examples of parsing mpegflow output. examples/vis_hue can also be used to produce hue flow visualizations like above.

mpegflow depends only on a recent FFmpeg, vis depends on FFmpeg, OpenCV and libpng. The tools are known to work with FFmpeg 3.1 and OpenCV 3.1.

Once the dependencies are visible to g++, run:

make     # to build mpegflow
make vis # to build vis

You will probably end up with a shared build, for static build, please feel free to play with Makefile.

You may use wigwam to install them to a local directory (no root required):

wigwam init
wigwam install opencv ffmpeg
wigwam in

To build the tools on Windows:

1. Install 7zip
2. Create directory dependencies
3. Extract these FFmpeg builds to the dependencies directory using 7zip (for mpegflow and vis): https://ffmpeg.zeranoe.com/builds/win64/dev/ffmpeg-3.0.1-win64-dev.7z and https://ffmpeg.zeranoe.com/builds/win64/shared/ffmpeg-3.0.1-win64-shared.7z
4. Extract this OpenCV build to the dependencies directory using 7zip (for vis): https://github.com/opencv/opencv/releases/download/3.1.0/opencv-3.1.0.exe
5. Open VS2015 x64 Native Tools Command Prompt (VS2015 Community Edition will work) and run:

nmake mpegflow.exe FFMPEG_DIR=dependencies\ffmpeg-3.0.1-win64-dev\ffmpeg-3.0.1-win64-dev
# nmake vis.exe OPENCV_DIR=dependencies\opencv-3.1.0\opencv\build\x64\vc14

6. The Windows build is not fully static. You need to keep avutil-54.dll, avformat-56.dll, avcodec-56.dll, swresample-2.dll (for mpegflow) and opencv_world310.dll (for vis) in the same directory as the binary. Note that the instructions and the Makefile assume x64 machine architecture.