Skip to content

GSV-Cities: a large-scale dataset for visual place recognition

Notifications You must be signed in to change notification settings

amaralibey/gsv-cities

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

34 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

GSV-Cities

Official repo for Neurocomputing 2022 paper GSV-Cities: Toward Appropriate Supervised Visual Place Recognition

[ArXiv] [ScienceDirect] [Bibtex] [Dataset]

  • The dataset is hosted on [Kaggle].
  • Training can be run from main.py, the code is commented and should be clear. Feel free to open an issue if you have any question.
  • To evaluate trained models, we rewrote the evaluation script for 8 benchmarks, check the included Jupyter Notebook

Summary of the paper

  1. We collected GSV-Cities, a large-scale dataset for the task of Visual Place Recognition, with highly accurate ground truth.
    • It contains ~530k images.
    • There are more than 62k different places, spread across multiple cities around the globe.
    • Each place is depited by at least 4 images (up to 20 images).
    • All places are physically distant (at least 100 meters between any pair of places).
  2. We proposed a fully convolutional aggregation technique (called Conv-AP) that outperforms NetVLAD and most existing SotA techniques.
  3. We consider representation learning for visual place recognition as a three components pipeline as follows:

pipeline

What can we do with GSV-Cities dataset and the code base in this repo?

  • Obtain new state-of-the-art performance.
  • Train visual place recognition models extremely rapidly.
  • No offline triplet mining: GSV-Cities contains highly accurate ground truth. Batches are formed in a traightforward way, bypassing all the hassle of triplet preprocessing.
  • Rapid prototyping: no need to wait days for convergence (expect 10-15 minutes of per epoch).
  • All existing techniques can benefit from training on GSV-Cities.

Trained models

Please refer to the following Jupyter Notebook for evaluation.

Backbone Output
dimension
Pitts250k-test Pitts30k-test MSLS-val Nordland
R@1 R@5 R@1 R@5 R@1 R@5 R@1 R@5
ResNet50 8192
[2048x2x2]
92.8 97.7 90.5 95.2 83.1 90.3 42.7 58.8 LINK
4096
[1024x2x2]
92.5 97.7 90.5 95.3 83.5 89.7 42.6 59.8
2048
[512x2x2]
92.3 97.5 90.6 95.1 83.4 90.3 40.3 56.6
512
[128x2x2]
90.7 96.6 89.1 94.6 82.6 90.0 36.3 53.1

Code to load the pretrained weights is as follows:

from main import VPRModel

# Note that these models have been trained with images resized to 320x320
# Also, either use BILINEAR or BICUBIC interpolation when resizing.
# The model with 4096-dim output has been trained with images resized with bicubic interpolation
# The model with 8192-dim output with bilinear interpolation
# ConvAP works with all image sizes, but best performance can be achieved when resizing to the training resolution

model = VPRModel(backbone_arch='resnet50', 
                 layers_to_crop=[],
                 agg_arch='ConvAP',
                 agg_config={'in_channels': 2048,
                            'out_channels': 1024,
                            's1' : 2,
                            's2' : 2},
                )


state_dict = torch.load('./LOGS/resnet50_ConvAP_1024_2x2.ckpt')
model.load_state_dict(state_dict)
model.eval()

GSV-Cities dataset overview

  • GSV-Cities contains ~530,000 images representing ~62,000 different places, spread across multiple cities around the globe.
  • All places are physically distant (at least 100 meters between any pair of places).

example

Database organisation

Unlike existing visual place recognition datasets where images are organised in a way that's not (so humanly) explorable. Images in GSV-Cities are named as follows:

city_placeID_year_month_bearing_latitude_longitude_panoid.JPG

This way of naming has the advantage of exploring the dataset using the default Image Viewer of the OS, and also, adding redondancy of the metadata in case the Dataframes get lost or corrupt.

The dataset is organised as follows:

├── Images
│   ├── Paris
│   │   ├── ...
│   │   ├── PRS_0000003_2015_05_584_48.79733778544615_2.231461206488333_7P0FnGV3k4Fmtw66b8_-Gg.JPG
│   │   ├── PRS_0000003_2018_05_406_48.79731397404108_2.231417994064803_R2vU9sk2livhkYbhy8SFfA.JPG
│   │   ├── PRS_0000003_2019_07_411_48.79731121699659_2.231424930041198_bu4vOZzw3_iU5QxKiQciJA.JPG
│   │   ├── ...
│   ├── Boston
│   │   ├── ...
│   │   ├── Boston_0006385_2015_06_121_42.37599246498178_-71.06902130162344_2MyXGeslIiua6cMcDQx9Vg.JPG
│   │   ├── Boston_0006385_2018_09_117_42.37602467319898_-71.0689666533628_NWx_VsRKGwOQnvV8Gllyog.JPG
│   │   ├── ...
│   ├── Quebec
│   │   ├── ...
│   ├── ...
└── Dataframes
    ├── Paris.csv
    ├── London.csv
    ├── Quebec.csv
    ├── ...

Each datadrame contains the metadata of the its corresponding city. This will help access the dataset almost instantly using Pandas. For example, we show 5 rows from London.csv:

place_id year month northdeg city_id lat lon panoid
130 2018 4 15 London 51.4861 -0.0895151 6jFjb3wGyCkcBfq4k559ag
6793 2016 7 2 London 51.5187 -0.160767 Ff3OtsS4ihGSPdPjtlpEUA
9292 2018 1 289 London 51.531 -0.12702 0t-xcCsazIGAjdNC96IF0w
7660 2015 6 97 London 51.5233 -0.158693 zFbmpj8jt8natu7IPYrh_w
8713 2008 9 348 London 51.5281 -0.127114 W3KMPec54NBqLMzmZmGv-Q

And If we want only places that are depicted by at least 8 images each, we can simply filter the dataset using pandas as follows:

df = pd.read_csv('London.csv')
df = df[df.groupby('place_id')['place_id'].transform('size') >= 8]

Notice that given a Dataframe row, we can directly read its corresponding image (the first row of the above example corresponds to the image named ./Images/London/London_0000130_2018_04_015_51.4861_-0.0895151_6jFjb3wGyCkcBfq4k559ag.JPG)

We can, for example, query the dataset with only places that are in the northern hemisphere, taken between 2012 and 2016 during the month of July, each depicted by at least 16 images.

Stay tuned for tutorials in the comming weeks.

Cite

Use the following bibtex code to cite our paper

@article{ali2022gsv,
  title={GSV-Cities: Toward appropriate supervised visual place recognition},
  author={Ali-bey, Amar and Chaib-draa, Brahim and Gigu{\`e}re, Philippe},
  journal={Neurocomputing},
  volume={513},
  pages={194--203},
  year={2022},
  publisher={Elsevier}
}