Is there a viable alternative to Coral?

[kspearrin]

I have been trying to find a Coral TPU for the past 4 months, but they are out of stock everywhere. Most of the stores I check don't show stock available until later this year. I would really like to adopt Frigate as an NVR, but my understanding is that I need a Coral. Is there a viable alternative to running Frigate in production with 10 cameras?

[jmorris644]

Some of us are using a Jetson Nano. But those are getting pricey as well.

[rgobbel]

Just started looking at Frigate today, and see this in the Detectors section of the documentation:

"There is no support for Nvidia GPUs to perform object detection with Tensorflow."

Has someone actually implemented NVIDIA GPU support, and if so, where's the code?

I have four NVIDIA Jetsons of various types, 3 of them just gathering dust at the moment, and I'd rather not have to buy a hardware device whose basic compute power is less than what I already have on hand. If it's not crazy amount of effort, I'd be willing to do the implementation myself, but if the basics are already there, that would obviously be preferred.

[NickM-27]

Jetson nano support is being added in #2548

[tube0013]

M1 based Macs also seem to work well. def high on the price range though.

[cloudandy]

@tube0013 except for the networking issues with Docker on MacOS…

[ireuben]

@cloudandy do you have any more details of the networking issues? Do they affect Frigate specifically? I've been running Frigate on Docker on macOS for a couple months now, and seems to be working fine.

[jvrobert]

If you have a reasonably fast machine multi-core CPU with hardware decode acceleration you may be alright, maybe just give it a try. I was able to get ahold of one of the m.2 dual tpu ones pretty easily, but my machine only uses one of the tpus. Some guy was making adapters to allow both tpus to be used but, umm, we may not be seeing products from the country I think he was from anytime soon.

[kbrouder]

I have been running cpu detectors for several weeks now and it works very well. I shut down my blue iris VM completely after testing for just a few days because it worked so well. To be honest, I was left scratching my head a bit as to why there are all the warnings about not using CPU detectors because my setup is working well and overall using less resources than my previous blue iris config.

I am running it in a dedicated Hyper-v ubuntu VM with docker compose on a dell poweredge R610 (very old server). I believe I have the VM set to 8-10vcpus and ~7-8gb memory. I have 7 cameras, 6 are recording 4k, streaming 2k, and detectors on 640p (hikvision), the last one is a 1080p wireless camera I havent replaced yet.

Probably not the most power efficient, but it works just fine. Most people probably don't have a hyper-v host server, and I would not recommend that as optimal, but many of the cheap mini pc's available today have celeron processors that are around the same performance as my old dell server (given it has 2 cpus). So anything with a decent processor I would imagine would do just as well. I tested a sub $200 mini pc and it was working just fine as well.

[kspearrin]

I just tried all 10 cameras using 4 CPUs and am getting inference speeds ~50. Not sure if this is good or not, but my overall host server is utilizing ~50-60% of overall CPU resources, so it seems sustainable.

@kbrouder what kind of inference speeds do you get?

[kbrouder]

@kspearrin that is very good I think, mine are more like 120-150. I am not sure I fully understand it, but I believe the math goes something like this, 1000/50 = 20 detection fps, so if your detect fps is 5 you could theoretically handle 4 cameras per cpu detector.

[kspearrin]

@kbrouder It seems that over time, my inference speeds go up. Currenting varying from 150-300 range. CPU usage has also gone up after 24 hours. Not sure. If I restart, they go back down. I also notice that my recorded video is quite choppy. I'll keep it going and monitor for a while.

Please do let me know if you want to get rid of the e-key coral that you have. I have an e-key slot on my server's motherboard, so I don't even need adapters for that model. From what I read, it appears I can passthrough to proxmox without issue.

[kbrouder]

@kbrouder It seems that over time, my inference speeds go up. Currenting varying from 150-300 range. CPU usage has also gone up after 24 hours. Not sure. If I restart, they go back down. I also notice that my recorded video is quite choppy. I'll keep it going and monitor for a while.

Please do let me know if you want to get rid of the e-key coral that you have. I have an e-key slot on my server's motherboard, so I don't even need adapters for that model. From what I read, it appears I can passthrough to proxmox without issue.

That's odd, possibly your motion settings are too sensitive? I will let you know, have another adapter and a jetson nano coming next week, so I'll be doing some fiddling with those.

[kbrouder]

@kspearrin - I am giving up on the dual edge tpu e-key coral. I cannot get any adapters to work to show even 1 TPU. PM me your shipping zip and ill sell it to you for what I am in it for....which I just did the conversion....$97 USD because I got it shipped from Germany...I'll glady PM you the receipt if you want it, not trying to make money on it but probably could if I listed it on Ebay so no biggie if you don't want to pay that much.

Edit - Do they have DMs on Github? Ill fill out the contact form on your website with my email address....

[blakeblackshear]

if your detect fps is 5 you could theoretically handle 4 cameras per cpu detector

That's not quite right. The fps of the detections is only indirectly related to the detect fps. Frigate often runs multiple detections on the same frame. It also doesn't run any detections on a frame when there is no motion detected. You will start to see skipped frames if the there is more activity than the detectors can handle.The Coral will bring inference times down to 10ms and remove all the CPU load related to running object detection.

At this point, there are no other supported hardware accelerators.

[blakeblackshear]

I already have one, so I can see what the effort would be. If it's easy enough, then I will add support.

[John8989doe]

I know it's been said there's no NVIDIA support but with NVIDIA open sourcing it's drivers is there any hope they can be utilized?

[NickM-27]

I know it's been said there's no NVIDIA support but with NVIDIA open sourcing it's drivers is there any hope they can be utilized?

Nvidia has already been utilized here: #3016

[jwshields]

According to this link: https://community.intel.com/t5/Intel-Distribution-of-OpenVINO/EOL-Movidius-Neural-Compute-Stick-2/m-p/1365062
NCS2, while EOL'd, will have technical support until June 2023, and warranty until June 2024. Is there still no chance of getting it integrated?

[NickM-27]

anyone with an NCS2 and running amd64 arch you can check out #3797

[RLutsch]

anyone try the intel movidius compute stick? seems to support tensorflow lite but documentation a bit sketchy.

[zombielinux]

I wonder if there is an FPGA implementation that could be used. Even if its less performant, there are a number of FPGA development boards (namely the ARTY board) that should be able to get the job done.

[Gerben321]

This sounds hopeful: https://9to5google.com/2022/05/05/google-coral-asus/

[DuncanCT]

Any update on:
Support for NCS2?
Coral stock anywhere?

[simeon-walker]

Today I had notification that my Farnell order has shipped despite the last update saying it would be another year. Maybe a sign that significant numbers are being made.

[NickM-27]

Yeah, Asus taking over manufacturing seems to have come to fruition.

[soubhik-khan]

https://www.newark.com/ has some stocks of coral.. 2-3months lead time

[NickM-27]

@MostHated If you still have NCS2, check out #3797

[MostHated]

@MostHated If you still have NCS2, check out #3797

👍

[TD22057]

There are a lot more for sale on ebay now (usb models) than there used to be at around $200. There was very little on ebay for awhile and now there are a lot so I'm hopeful this means that more are entering the market and prices will come down.

[NickM-27]

I've noticed the same, on Amazon as well

[asantaga]

just bought at USB-C version at RS Components https://uk.rs-online.com/web/p/development-tool-accessories/2017821

[kbrouder]

annoying they dont list it on their export site (ship to USA)

[TD22057]

@kbrouder You can order them from Newark's US site and they'll ship from UK stock. I put an order in on July 21'st for the USB model. It was a little price ($130 after taxes and shipping) but better than ebay. Current status says it will ship Aug 20th but we'll see.

[MostHated]

No such luck with shipping yet : /

[TD22057]

Nope. I just had them check my order which was supposed to ship last week and they said they have no idea. I'm cancelling. Prices on amazon and ebay are coming down below $150 now. I'll probably just get one from those sources when I know I have time to work on it.

[warlordattack]

any chance to find a Coral for France somewhere please ? :)

[tsrubar]

Just got USB Coral from here: https://botland.cz/google-coral/16911-google-coral-usb-accelerator-akcelerator-edge-tpu-ml-arm-cortex-m0-193575021935.html
There are 44 pieces available at the moment.
Ordered 2 days ago, today arrived.

[geerlingguy]

It's crazy how the Coral is still so low in stock everywhere; at least scalpers are only getting around $100 now lol.

I also found out about the Hailo AI accelerator recently... it may be fairly pricey though, too.

[Shred99]

The Hailo AI might be an interesting alternative.

I've been using an Intel Neural Compute Stick 2 for quite a while now. It's not as powerful as the Coral, but at least it was available at the time. It's a shame that Intel discontinued it in June 2023.

[MostHated]

I had been on waiting lists/preorders for them a while back (initially signed up early 2022, I believe, received them early 2023), and was just going to cancel whichever ones were not the first to come in stock, and they all happened to have come in stock at the same time. Now I have 3, lol. The 4gb version of the Jetson Nano is about $150, while I don't recall if it is compatible with this, I have been using it as a remote Deepstack processor, and it works well, too.

[The-NinthWave]

I am running a Quadro P600 and TensorRT gives me an inference of 10,5 mS
This card draws max 30W, can trancode my Jellyfin streams and detect in Frigate.
Pretty nice to me

[askpatrickw]

#11961 <-- this would be an option.