[Question]: Information about skipped frames

[Sch0field9]

What is your question:

Hi there, I was wondering if there is some infromation around skipped frames and how grave they are.

I'm asking because I sometimes have cameras with 20 detect fps and maybe 5 skipped, but on the other hand I also have a cameras at the same time with 35 detect fps and only 1-3 skipped.

Also what is the cause of skipped frames?
I have 2 pcie TPUs they are hovering around 5ms and 17-18% CPU usage( Which I assume is CPU usage from the Coral) . No matter the detect frames and no matter if I use 1 or 2 Corals. What is most likely the bottle neck here? CPU speed, thread count, memory speed or storage speed?

And lastly. Is my assumption correct that skipped frames can cause the model to be less accurate? Reasoning that if you take the median of the last 3 frames and one of them gets skipped, it has a lower chance to detect that object. Or is this neglectible?

I know 30-40 detect fps is high. But in this season we have strong winds, which move big branches and whole trees. Which are also in the area I want detection.

[NickM-27]

Frigate prioritizes real time detection. If the object detection process has too many detections waiting in the queue then it must skip that detection to remain real time. This means object tracking may become confused and some objects may be missed.

I have 2 pcie TPUs they are hovering around 5ms and 17-18% CPU usage( Which I assume is CPU usage from the Coral)

no, this is the CPU usage of the process that controls the coral. Corals do not report "usage" because that does not exist.They just run an inference one at a time.

I know 30-40 detect fps is high. But in this season we have strong winds, which move big branches and whole trees. Which are also in the area I want detection.

I don't see any reason that this would warrant running higher detect fps. I really see no reason detect fps would ever need to be more than 10 in the vast majority of cases

[Sch0field9]

Thank you for your reply. This clears up a some confusion.
to specify: My detect fps is set to 5 or 10 fps depending on the camera and how fast object pass these cameras.
The system report shows then the 15-40 detect fps. I thought that this means 1 frame can have multiple detections at once, and therefore reporting detect fps above the defined detection fps.
Is this not expected behavior?

Because strong winds move whole trees and branches, which are also in the ROI, these trigger lots of motion to analyze.

[NickM-27]

I misunderstood what you meant by that, I thought you meant you were setting detect fps that high. Yes, in cases where there is a lot of activity some skipped frames is okay and it sounds like your setup is done correctly

[Sch0field9]

Ok thanks. I just was confused bc sometimes it says 23 (5 skipped) and sometimes 35 ( 0.1 skipped) for example.
Then I suppose it depends on the total amount of detection fps at that moment and a bit of luck which frames are skipped.
So it would indeed be convenient to also have the total detect fps in the systems tab. But as far as I know this is already an open issue and is in the works.
I'm already exited for the 0.14-beta release.

[NickM-27]

Right, it's a closed issue because it has been implemented in 0.14