MONAI Geometric Workflow

[vikashg]

Is your feature request related to a problem? Please describe.
An end to end use case for MONAI Geometric workflows
Describe the solution you'd like
If we are considering a pipeline which will have the most impact on the MONAI Community and will draw more attention to the work we are doing, it will be adding a detection pipeline. The way I envision it is that we can make a big claim in the next release that
"MONAI now supports Detection workflows".
Some of the use cases for detection are

• Device Detection in chest X-rays
• Lung Nodule detection in chest X-rays
• Fracture detection

These use cases are generally 2D in nature and just by the default nature of the annotation tools being used and the public dataset available, two things happen

1. These image data is generally in JPEG format
2. The annotations are by default recorded in image space at subpixel accuracy. This means that the World Coordinate System and the Image Index system share the same default origin (0, 0) and image spacing (1, 1). Further, the annotation might be recorded as (12.32, 24.67) instead of (12, 24). The annotation do not generally snap to the closest index.

Additional context
We should develop a pipeline based on Microsoft coco object detection dataset. We should have no problems building this out on MONAI frameworks as we can load JPEG images using our LoadImage function. Once we build the pipeline we can replace the JPEG image with our medical JPEG images.

[vikashg]

https://github.com/Project-MONAI/MONAI/blob/dev/monai/apps/detection/transforms/box_ops.py

[vikashg]

Another example from @ericspod https://github.com/Project-MONAI/model-zoo/tree/dev/models/valve_landmarks

[25benjaminli]

Interestingly, MONAI has a tutorial for 3D detection but I'm hesitant to say whether it has 2D support. In any case, I was able to achieve moderate accuracy and performance by modifying the Luna16 dataset processing and 3D RetinaNet for a different medical detection task, and I believe the pipeline could be modified similarly for 2D images.

However, it was certainly a bit of a headache manually changing all the dataset processing, transforms, etc. It would be nice to have a more unified pipeline that follows the COCO format, as you mentioned, since it's not only standardized but also easier to work with. Also, adding options for more models would certainly improve MONAI's reach.

[mingxin-zheng]

One usecase for geometric transform is to process US 2D images with point annotations on the vessel walls and no label.

• Inputs
  • 2D vessel images showing the transverse view of a vessel
  • 3 points that the annotator picked on the rounded vessel wall
• Expected result
  • train a neural network to segment the vessel wall in the transverse view
• Potential use of the geometric transform
  • In the pre-processing transforms, one custom transform is introduced to fit a circle from the 3 clicked points, and generate more points as the vessel wall
  • Later, these interpolated points may be used to generate label masks of the vessel wall.