[DEV] Yolov8 Models

[alexyao2015]

The post processing done on the yolov8 model requires nms to be applied to the boxes. I've noticed that with the yolov8_postprocess method, the cv2 NMS doesn't work properly. As a test, I removed the mask on the arrays and passed everything to the cvs.dnn.NMS to see if everything works correctly, but it only returns a single result when it should return 10 using a mask. It seems to me that something must be wrong with the cv2.dnn.NMS usage.

When looking at the yolov8 source, we see it uses this nms method to perform the post processing. I wonder there is objection to either installing the ultralytics library or simply reimplementing this nms method into the frigate utilities. We can also consider simply adding a yolov8 plugin and using the ultralytics package directly since this provides all the utilities for running across any backend already rather than reimplementing all these different backends into frigate manually. I think this would also provide users with some

TLDR: The nms method currently in frigate doesn't work properly.

CC: @harakas as the original implementer of the yolov8_postprocess method

[harakas]

Can you please offer an image+raw data+processed data+result.

[alexyao2015]

Thats weird. I can't seem to reproduce the issue I was having yesterday now, so problem solved I suppose. About this method, why is there a mask being applied instead of just running NMS on the boxes always? Is it more performant to attempt masking then NMS?

[harakas]

Mainly the mask in yolov8_postprocess it is used to limit the number of boxes returned as frigate plugin api only supports 20 detections and so we filter out low probability boxes that are not used anyways. cv2.dnn.NMSBoxes is only called when the returned box count would be higher. Frigate itself also calls the cv2.dnn.NMSBoxes later on when processing the detections.

[NickM-27]

Having worked with the nms method in other parts of the frigate codebase, I can say that it works properly according to the cv2 documentation