By Zhuoyi Zhao and Chengyan Fan from Cryosphere Lab, ESSC, CUHK.
Geo-SAM is a QGIS plugin that aims to help people segment, delineate or label landforms efficiently when using large-size geospatial raster images. Segment Anything Model (SAM) is a foundation AI model with the superpower, but the model size is huge, and using it to process images can take a long time, even with a modern GPU. Our tool uses the strategies of encoding image features in advance and trimming the SAM model. The interactive segmentation algorithm can perform real-time inference (at millisecond speeds) on a laptop CPU, making it a convenient and efficient tool for dealing with remote sensing images.
The Geo-SAM plugin includes two separate parts: the Image Encoding Part, and the Interactive Segmentation Part. The image encoding part is designed to generate and save the image features using the SAM image encoder, and the encoding process only needs to run once per image. The segmentation part is for interactively segmenting landforms, and it can only be used to segment preprocessed images (whose features have been generated in advance using the encoding tool, as the included demo image).
 |
|---|
Comparison of the workflow between Geo-SAM and the original SAM. original SAM package encodes prompts and image simultaneously, while the Geo-SAM model encodes image into feature files at once and queries prompts in real-time (at millisecond speeds) by loading those saved features. |
Read the documentation for more details: https://geo-sam.readthedocs.io/en/latest/.
- Interactive Segmentation Demo (millisecond-level response time after providing prompts)
Preview ModeDemo for Interactive Segmentation (Execute SAM following the mouse cursor)
- Image Encoder Demo (QGIS plugin part)
- Encoder Copilot Demo
- Based on QGIS for a user-friendly GUI and cross-platform compatibility without programming skills needed.
- It provides segmentation results
instantly after giving prompts, and can even display resultsin real-time following the mouse cursor(Preview mode, currently only available in the dev version, will be added to the stable version after being rigorously tested). Users can have a smooth, interactive experience.This can greatly improve the efficiency and user experience of segmentation.
- SAM is designed to segment one object once with a series of prompts, so you should save the current results before getting to the next one when using the Geo-SAM tool.
- SAM natively supports only three-band images, but we have adapted Geo-SAM to support one or two-band images so that you can try grayscale images, spectral index images (like NDVI, NDWI), or even SAR images.
- The Geo-SAM plugin is currently in active development. We will continue making improvements and welcome your feedback. If you have any questions or suggestions, please feel free to open an issue or discussion on our GitHub repository at GitHub Issues or GitHub Discussions.
Zhao, Zhuoyi, Fan, Chengyan, & Liu, Lin. (2023). Geo SAM: A QGIS plugin using Segment Anything Model (SAM) to accelerate geospatial image segmentation (1.1.0). Zenodo. https://doi.org/10.5281/zenodo.8191039
@software{zhao_zhuoyi_2023_8191039,
author = {Zhao, Zhuoyi and Fan, Chengyan and Liu, Lin},
title = {{Geo SAM: A QGIS plugin using Segment Anything Model (SAM) to accelerate geospatial image segmentation}},
month = jul,
year = 2023,
publisher = {Zenodo},
version = {1.1.0},
doi = {10.5281/zenodo.8191039},
url = {https://doi.org/10.5281/zenodo.8191039}
}This repo benefits from Segment Anything and TorchGeo. Thanks for their wonderful work.