Laboratory of Biomechanical and Image Guided Surgical Systems, Johns Hopkins University
- 3 View Inference
- Embedding saving
- Data type
- Volume
- 2D Image
- RGB Image (WIP)
- models
- interactions
- positive and negative points
- 2D bounding box
- 3D bounding box (WIP)
- combination
- automatic segmentation
- training (WIP)
Smaller models may have worse accuracy.
https://www.youtube.com/watch?v=vZK45noZVIA
Accurate image segmentation is crucial for medical image analysis as it enables clinicians to extract meaningful information from the image. It also allows for the detection of subtle changes in the tissue or organ of interest, which is essential for monitoring disease progression and treatment response.
Our project, Segment Any Medical-Model aims to develop an integration for Segment Anything Model (SAM) and 3D Slicer for future development and validation of the potentials of transferring Large Language Model to the medical image analysis field.
Works both on Linux and Windows. Has Mac support, but not tested yet.
This assumes Cuda, cv2 and pytorch are in your environment.
git clone [email protected]:bingogome/samm.git
conda create --name samm
conda activate samm
pip install git+https://github.com/facebookresearch/segment-anything.git
pip install opencv-python pycocotools matplotlib onnxruntime onnx
If you are using Windows, it's okay if you don't install pycocotools.
Start 3D Slicer, in the Python Console:
slicer.util.pip_install("pyyaml")
slicer.util.pip_install("pyzmq")
slicer.util.pip_install("tqdm")
SD Slicer -> Developer Tools
→ Extension Wizard
.
Extension Tools
-> Select Extension
-> import the samm/samm folder.
Back to terminal, cd to samm (upper level)
Run ./samm-python-terminal/sam_server.py
If it throws an error missing "sam_vit_h_4b8939.pth", move segment-anything/notebooks/sam_vit_h_4b8939.pth to samm/samm-python-terminal/samm-workspace
Follow the demo and Segment Any Medical Model away!
Install this repo:
git clone [email protected]:bingogome/samm.git
It's essential to have a clean virtual environment to avoid any potential conflicts. Therefore, you'd better to create a new environment for running the rest part of the code.
Install any version of anaconda to manage the virtual environment. Anaconda installation guide can be found here.
Create virtual environment and activate it:
conda create --name samm
conda activate samm
Note: The given python script in this folder has to be executed in samm venv.
The code requires python>=3.8
, as well as pytorch>=1.7
and torchvision>=0.8
. Please follow the instructions here to install both PyTorch and TorchVision dependencies. Installing both PyTorch and TorchVision with CUDA support is strongly recommended.
Install Segment Anything:
pip install git+https://github.com/facebookresearch/segment-anything.git
or clone the repository locally and install with
git clone [email protected]:facebookresearch/segment-anything.git
cd segment-anything; pip install -e .
Then,
pip install opencv-python pycocotools matplotlib onnxruntime onnx
Follow this page to download a compatible version of 3D Slicer and install it in your local environment.
The source code of the extension is contained in samm.
In the GUI of 3D Slicer, expand the extension drop-down menu, and choose Developer Tools
→ Extension Wizard
.
Then on the left side of the GUI, click the toggle bar named Extension Tools
and click `Select Extension' button. It will prompt a navigation window where you can find, select and import the samm folder.
If you use SAMM in your research, please consider use the following BibTeX entry.
@article{liu2023samm,
title={SAMM (Segment Any Medical Model): A 3D Slicer Integration to SAM},
author={Liu, Yihao and Zhang, Jiaming and She, Zhangcong and Kheradmand, Amir and Armand, Mehran},
journal={arXiv preprint arXiv:2304.05622},
year={2023}
}