A unified API for quickly and easily trying 29 (and growing!) image matching models.

Compare matching models across various scenes. For example, we show SIFT-LightGlue and LoFTR matches on pairs:

(1) outdoor, (2) indoor, (3) satellite remote sensing, (4) paintings, and (5) a false positive.

To install this repo run

git clone --recursive https://github.com/gmberton/image-matching-models

You can install this package for use in other scripts/notebooks with the following

cd image-matching-models
python -m pip install -e .

You can use any of the matchers with

from matching import get_matcher

device = 'cuda' # 'cpu'
matcher = get_matcher('superglue', device=device)  # Choose any of our ~20+ matchers listed below
img_size = 512

img0 = matcher.load_image('path/to/img0.png', resize=img_size)
img1 = matcher.load_image('path/to/img1.png', resize=img_size)

result = matcher(img0, img1)
num_inliers, H, mkpts0, mkpts1 = result['num_inliers'], result['H'], result['inliers0'], result['inliers1']
# result.keys() = ['num_inliers', 'H', 'mkpts0', 'mkpts1', 'inliers0', 'inliers1', 'kpts0', 'kpts1', 'desc0', 'desc1']

You can also run this as a standalone script, which will perform inference on the the examples inside ./assets. It is possible to specify also resolution and num_keypoints. This will take a few seconds also on a laptop's CPU, and will produce the same images that you see above.

python main.py --matcher sift-lg --device cpu --log_dir output_sift-lg

where sift-lg will use SIFT + LightGlue.

The script will generate an image with the matching keypoints for each pair, under ./output_sift-lg.

You can choose any of the following methods (input to get_matcher()):

Dense: roma, tiny-roma, dust3r, mast3r

Semi-dense: loftr, eloftr, se2loftr, aspanformer, matchformer, xfeat-star

Sparse: [sift, superpoint, disk, aliked, dedode, doghardnet, gim]-lg, dedode, steerers, dedode-kornia, [sift, orb, doghardnet]-nn, patch2pix, superglue, r2d2, d2net, gim-dkm, xfeat, omniglue

All the matchers can run on GPU, and most of them can run both on GPU or CPU. A few can't run on CPU.

To use on your images you have three options:

1. create a directory with sub-directories, with two images per sub-directory, just like ./assets/example_pairs. Then use as python main.py --input path/to/dir
2. create a file with pairs of paths, separate by a space, just like assets/example_pairs_paths.txt. Then use as python main.py --input path/to/file.txt
3. import the matcher package into a script/notebook and use from there, as in the example above

Our implementation of Patch2Pix (+ Patch2PixSuperGlue), R2D2, and D2Net are based on the Image Matching Toolbox (IMT). LoFTR and DeDoDe-Lightglue are from Kornia. Other models are based on the offical repos above.

Runtime benchmark is the average of 5 iterations over the 5 pairs of examples in the assets/example_pairs folder at image size 512x512. Benchmark is done using benchmark.py on an NVIDIA RTX A4000 GPU. Results rounded to the hundredths place.

* LightGlue model runtimes are listed in the order: SIFT, SuperPoint, Disk, ALIKED

See CONTRIBUTING.md for details.

This repo was created as part of the EarthMatch paper. Please consider citing EarthMatch work if this repo is helpful to you!

@InProceedings{Berton_2024_EarthMatch,
    author    = {Gabriele Berton, Gabriele Goletto, Gabriele Trivigno, Alex Stoken, Barbara Caputo, Carlo Masone},
    title     = {EarthMatch: Iterative Coregistration for Fine-grained Localization of Astronaut Photography},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
    month     = {June},
    year      = {2024},
}