Multiple Google Coral TPUs

[alex-savin]

Hey,

Is it possible to use multiple TPUs?!

Thanks,
Alex

[blakeblackshear]

It is possible, but I haven't tried it. I may have to make some modifications for Frigate to leverage both. https://coral.ai/docs/edgetpu/multiple-edgetpu/

[alex-savin]

@blakeblackshear I can the first one to try it
=)

[zacs]

@alex-savin any luck with this? Would love to throw another unit at this :)

[blakeblackshear]

How many cameras do you have? I am running 7 cameras at 5fps and I am only using ~20% of a single coral at peak times.

[zacs]

That's good news. I've got 4 cameras running also 5fps, and am also hovering around 20% on a 9th gen i5. I'll be pushing it up to 11 cameras and figured I may want a 2nd Coral. Glad to hear I may not need it. I'll ping back if I notice any perf issues going up to 10.

As an aside: thanks for Frigate. It is ridiculously turnkey and very, very nice to use.

[scantineau]

How many cameras do you have? I am running 7 cameras at 5fps and I am only using ~20% of a single coral at peak times.

What tool do you use to know the coral utilisation ?

[blakeblackshear]

I am just looking at data from frigate. You can calculate the max possible FPS of the coral with 1000/inference_speed. With 8-10ms inference speeds, you should be able to get over 100FPS (and I have hit that max in testing). Monitoring real world usage, I rarely see more that 20fps of processing on the coral.

[Timminater]

How about using the second coral with a different model? So object detection on coral1, and for example when a person is detected face recognition on coral2. I think this way the first coral can keep checking all cameras for models without interception and model change and the second coral can do specific model loading based on the results on coral1. This way we can fully use the https://coral.ai/products/m2-accelerator-dual-edgetpu/

[sygys]

isnt the amount of FPS a coral can process dependent of how high the resolution is? I guess 7 feeds on 1080p isnt the same as 5 feeds in 4K

[NickM-27]

isnt the amount of FPS a coral can process dependent of how high the resolution is? I guess 7 feeds on 1080p isnt the same as 5 feeds in 4K

The coral itself has no correlation with resolution. Every coral detection is run on an image of a specific size (defined by the model). In this case by default 320 x 320 pixels.

Frigate takes a region of motion / interest that it calculates and resizes this to the models size and then sends it to the coral for detection. Higher resolution will certainly impact the CPU so the CPU needs to be up for the task, but the need for corals has to do with fps having detection run and not resolution.

[sygys]

Thanks. always good to know how things work. So if im correct on a camera with loads of motion boxes the coral needs to do allot of processing?

I ask this because 2 of my 7 cameras have a ton of motion boxes due to foliage moving all the time in the wind. the red motion boxes in the debug section of frigate go wild. It seems my coral runs around 28% if i calculate correctly.

[NickM-27]

Thanks. always good to know how things work. So if im correct on a camera with loads of motion boxes the coral needs to do allot of processing?

I ask this because 2 of my 7 cameras have a ton of motion boxes due to foliage moving all the time in the wind. the red motion boxes in the debug section of frigate go wild. It seems my coral runs around 28% if i calculate correctly.

yes exactly, this is one of the key cases where motion masks come in to play to reduce these motion boxes in places where it is unlikely to be an object of interest. (you may know this, but figured I'd mention just in case).

[sygys]

Yes im starting to understand how this all works now. One last thing. if we talk about FPS a coral can process, this means when ever a bounding box is shown in the debug of frigate this bounding box is processed and this counts as 1 FPS right?

So this also explains the CPU usage of my NUC going at almost 30% when frigate is running. Because the motion detection is not done by the coral but the cpu of the system itself

[sygys]

im glad my watercooled system is ready to ship any moment now. to keep my NUC cool

The acrylic is being polished now. I hope it will ship tomorrow

[NickM-27]

Yes im starting to understand how this all works now. One last thing. if we talk about FPS a coral can process, this means when ever a bounding box is shown in the debug of frigate this bounding box is processed and this counts as 1 FPS right?

So this also explains the CPU usage of my NUC going at almost 30% when frigate is running. Because the motion detection is not done by the coral but the cpu of the system itself

Every time a motion box is shown that is a frame that the coral processes. It will also process frames in other circumstances once an object is being tracked.

But yes, all the motion detection / frame manipulation is handled by the CPU. You can see this more specifically in the debug section of frigate.