Add segment line predictions

[glenn-jocher]

@AyushExel @Laughing-q I'm adding this for outputting segment predictions outlines. I'm running into issues when objects are composed of multiple segments. What's been the COCO segment strategy for multi-part objects up until now?

EDIT: I used this tutorial to get started. Contours will be returned as a list, and I've concatenated them together for outputting and plotting, but this causes issues when cv2 breaks a prediction into multiple subsegments.

Signed-off-by: Glenn Jocher [email protected]

🛠️ PR Summary

_{Made with ❤️ by Ultralytics Actions}

🌟 Summary

Upgraded segmentation output with polygon support for more accurate object outlines. 🎨

📊 Key Changes

• Added a new function masks2segments to convert masks to polygon segments.
• Updated prediction code to rescale boxes according to image size and implement segmentation polygon drawing.
• Enhanced file output to include polygon segment data when saving to .txt.

🎯 Purpose & Impact

• Accuracy: Polygon segments offer more precise object outlines compared to bounding boxes, improving detection accuracy. 🎯
• Utility: Enables saving detailed object shape information, which can be crucial for certain applications like medical image analysis or precision agriculture. 🔍
• Visualization: Users can now visualize object outlines with polygons directly on images, making the results more intuitive and informative. 🖼️

This update enhances the YOLOv5 model's ability to not only detect but also delineate object shapes more precisely, which could significantly benefit applications that require detailed object recognition.

[AyushExel]

@glenn-jocher This is the current stragetegy to merge multi-part segments implemented by @Laughing-q
https://github.com/ultralytics/JSON2YOLO/blob/e6b0a258d8abc0b31cc6e4ad13a87bd8610724ec/general_json2yolo.py#L324-#L371
The algorithm is like this:

• Find the shortest path between the segments
• Connect the segments with a thin line( doesn't really visually affect the outputs)

[glenn-jocher]

@AyushExel ah got it. Perhaps we should duplicate merge_multi_segments into YOLOv5 then as well. In the past I've inserted None between breaks in the segments. This works for plotting as the None segments are usually ignored by matplotlib etc, but for --save-txt annotations I'm not sure how to handle these multi-part instances.

[AyushExel]

@glenn-jocher merging the merge_multi_parts and running the model results through it should solve the problem I think.
Initially we wanted to merge the coco2json file for 6.3 release but we changed course a bit. Do you want to do that here or just merge that one function?

[Laughing-q]

@glenn-jocher That's nice! I was thinking about doing the same thing, I've seen some issues there and the users seem to want this feature.

[glenn-jocher]

Ok I think this is fine for now. I experimented a bit and found that simply selecting the largest segment yielded the best results, at least on our test images.

We don't want to duplicate code as it's already causing many headaches for maintenance, so let's leave the JSON2YOLO code there.

The segment generation takes about 1ms per image on our test images, so it's only run if --save-txt is used, in which case the segment labels are saved there.

Plotting is unaffected (below images are only for debugging).