TensorRT using Nvidia GPU #3016
Replies: 66 comments 94 replies
-
|
Hi guys, I'm having some problems， |
Beta Was this translation helpful? Give feedback.
-
|
Did you use my modified files? That error looks like it's using raspberry pi files, which my modded files shouldn't be using. |
Beta Was this translation helpful? Give feedback.
-
|
I would love to try this as I too have a Quadro P400 that I'm just using for hardware decode at the moment, and I can't get hold of a Coral. However, I'm more of a docker user than a docker builder so I'm finding your instructions are a bit too vague for me to figure out what I need to do. Would be really great to see all the commands you used here, especially for the docker builds. |
Beta Was this translation helpful? Give feedback.
-
|
@yeahme49 great job! I'm successfully run Frigate with my GeForce GTX 750 Ti 2GB by your instruction. For a future build, we need to describe it in more details. |
Beta Was this translation helpful? Give feedback.
-
|
I've uploaded my latest image which should have all the latest 0.11.0 changes in it to docker hub. Should be able to use yeahme49/frigatetensor:latest to try this out without having to build the image yourself. Let me know how it works. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you! I will try this out and report the feedback on i3-8100, 8GB Ram and Nvidia GT 1030 |
Beta Was this translation helpful? Give feedback.
-
|
Can you please share any sample config for 1 or 2 camera setup? Edit: Never mind found it below! |
Beta Was this translation helpful? Give feedback.
-
|
@yeahme49 Awesome! Actually, I'm running on an unRaid host I had to quickly use Did I miss something? Or maybe you can update your docker image to create the dir? |
Beta Was this translation helpful? Give feedback.
-
|
I believe you need to map that directory on the host side to the container, the container shouldn't be the one creating that. |
Beta Was this translation helpful? Give feedback.
-
Good thinking! So obvious. Thanks for the suggestion - worked a treat. |
Beta Was this translation helpful? Give feedback.
-
|
I didn't test any gstreamer stuff since I thought that was targeted to the
Nvidia Jetson stuff and this is for PC so it might be missing. I'll look
into that
…On Thursday, June 23, 2022, urbydoo ***@***.***> wrote:
This camera config:
northwest:
gstreamer:
video_format: video/x-h265
audio_format: audio/x-alaw
inputs:
- path:xxxxx
is causing:
------------------------------
*** Config Validation Errors ***
------------------------------
argument of type 'NoneType' is not iterable
[2022-06-23 15:16:19] frigate.gstreamer ERROR : gst-inspect-1.0 failed
with the message: Traceback (most recent call last):
File "/opt/frigate/frigate/gstreamer.py", line 73, in
gst_inspect_find_codec
data = sp.check_output(
File "/usr/lib/python3.9/subprocess.py", line 424, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.9/subprocess.py", line 505, in run
with Popen(*popenargs, **kwargs) as process:
File "/usr/lib/python3.9/subprocess.py", line 951, in *init*
self._execute_child(args, executable, preexec_fn, close_fds,
File "/usr/lib/python3.9/subprocess.py", line 1823, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'gst-inspect-1.0'
Traceback (most recent call last):
File "/opt/frigate/frigate/app.py", line 313, in start
self.init_config()
File "/opt/frigate/frigate/app.py", line 83, in init_config
self.config = user_config.runtime_config
File "/opt/frigate/frigate/config.py", line 1088, in runtime_config
camera_config.create_decoder_cmds()
File "/opt/frigate/frigate/config.py", line 691, in create_decoder_cmds
gst_cmd = self._get_gstreamer_cmd(self.gstreamer, input)
File "/opt/frigate/frigate/config.py", line 759, in _get_gstreamer_cmd
get_gstreamer_builder(self.detect.width, self.detect.height, self.name)
File "/opt/frigate/frigate/gstreamer.py", line 356, in
get_gstreamer_builder
if builder.accept(available_plugins):
File "/opt/frigate/frigate/gstreamer.py", line 336, in accept
if plugin not in plugins:
TypeError: argument of type 'NoneType' is not iterable
------------------------------
*** End Config Validation Errors ***
------------------------------
Did i botch my config or is g-inspect missing from the container?
—
Reply to this email directly, view it on GitHub
<#3016 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AN6Z4QD4E6GEQA64DE2SSPTVQTQ5RANCNFSM5RVRZWOQ>
.
You are receiving this because you were mentioned.Message ID:
***@***.***
com>
|
Beta Was this translation helpful? Give feedback.
-
|
my fault, i'll flip back to fmpeg, i was going down that path given you started with that fork. Is this the right detector setup? |
Beta Was this translation helpful? Give feedback.
-
|
This is what I have and This image probably won't work with a jetson even with ffmpeg since it was built for amd64 not arm, but I could be wrong. |
Beta Was this translation helpful? Give feedback.
-
|
It works! Thank you so much @yeahme49! I'm using a Xeon D-2146NT, 64GB RAM and a Quadro P400. For reference, here's my minimal setup: docker-compose.yml: config.yml: which gives me the following output from Frigate debug stats: and I have a screenshot of me surrounded by a bounding box labelled 'person', but I'm not gonna post that... |
Beta Was this translation helpful? Give feedback.
-
|
FYI: In your docker-compose.yaml, you have two blocks defining environment variables . I didn't notice the second environment definition at the bottom and was getting some weird behavior where newly added environment variables the first block (i.e. setting YOLO_MODELS=XXX) weren't getting set in the container (even though the original NVIDIA environment variables from your docker-compose were). Moving all the environment variables to a single block fixed it. Just pointing it out to potentially save someone else some head scratching. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for pointing this out. I actually noticed a week ago when I was trying to add my Frigate+ API key and got the same behaviour as you. But then I forgot to update my post here. I suspect the NVIDIA_* variables are redundant as something, possibly |
Beta Was this translation helpful? Give feedback.
-
Here are the docs for the 0.11 beta release which don't include that anymore |
Beta Was this translation helpful? Give feedback.
-
|
Getting some errors the pre-built image (yeahme49/frigatetensor:latest): Seems like I have a way too fresh version of CUDA? |
Beta Was this translation helpful? Give feedback.
-
|
What GPU do you have and what version of compute does it support? Might be that it's older than the Quadro P400 that @yeahme49 presumably built for. I have v1.10.0-1 of nvidia-container-toolkit, which works fine, so it's probably not that yours is too new. For reference, here are my nvidia packages on Ubuntu Server 20.04: |
Beta Was this translation helpful? Give feedback.
-
|
Is it possible to just post a layman step by step installation instructions? It will help newbies like us. |
Beta Was this translation helpful? Give feedback.
-
|
Can you pull the docker image again? I rebuilt it for compute 3.5 and up now instead of just 6.1 so hopefully it should work. |
Beta Was this translation helpful? Give feedback.
-
|
Getting a slightly different error, too low of a version. |
Beta Was this translation helpful? Give feedback.
-
|
Yeah, this is absolutely phenomenal so far. I have used frigate 2 years, across 2 installations for 45 cameras and 7 tpu. I just replaced my home setup that was on two corals running 10 cameras and shifted to my quadro P2000. Im getting stellar performance with the yolov4 416 tiny. 4.8 ms or so. I am giddy as the model options are far wider. Next ill be deploying it at my business thats currently running 5 corals across 26 cameras. I just bought an nvidia A30 a few days ago to swap out the corals and move to use tensorrt on another platform. Very happy i can potentially stay with frigate. @blakeblackshear hopefully you can investigate see for yourself, and hopefully I dont speak to soon. This could give you far greater model flexibility as you move frigate+ forward. Currently my quadro p2000 is showing about 6 to 8 percent of gpu memory being used per 1920p 5 to 7fps camera running detection. @yeahme49 If your ever in Oklahoma, I owe you a few brews. |
Beta Was this translation helpful? Give feedback.
-
|
and I meant to add that due to probably cpu, memory bottleknecks, as well as i have another quADRo decoding at my work, the 5 tpus have a rough time with 26, 8 fps cameras. im constantly tweaking motion, various things to reign in detection fps to try and get not waste those precious bounding boxes. This is what I hope have solved. Keeping all decoding, detection, ect on the card to stop these slow downs if possbile. |
Beta Was this translation helpful? Give feedback.
-
|
So it looks like the tensorrt engine will only work on the same generator of card as what i have (so anything pascal) and won't work with anything else. I'll try to make the image build the engine on first run on each machine so that it builds properly or have a second image that does the building of the engine and you can copy the files over. It may take a bit though. So as of now it appears this will only work on pascal cards. |
Beta Was this translation helpful? Give feedback.
-
|
No worries. Appreciate everything so far. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for the great work! |
Beta Was this translation helpful? Give feedback.
-
|
@yeahme49 Just wanted to reach out say thank you! I've been desperately waiting for a back-ordered Coral, but I've now got your image successfully running on my laptop with GeForce MX150 card. I still need to do some tuning on the detection, and am having issues with hwaccel (MX150 doesn't do NVENC; hoping I can eventually figure out a way I can pass in my other intel gpu and use vaapi for the decoding/encoding part), but the detection is successfully running in GPU now. Thanks so much! |
Beta Was this translation helpful? Give feedback.
-
|
you can specify the decoder per input. I was able to setup deploy to my GPU server, using an nvidia A30 to do detection as it also doesnt have nvenc for some reason only decoders. The gpu "device" number for detections seems to be coded into the layer file or trt model file generated when you first load up. From there just specify those hwaccel options in the config. I dont see why this wouldnt work as im using an P400 and P600 to decode 26 odd camera streams and then the A30 to run inference from that. It adds some over head as it goes from p400/p600 -> cpu > A30. Since its your cpu you may not see any added latency |
Beta Was this translation helpful? Give feedback.
-
|
Can I try it on a Turing card (GTX 1650)? |
Beta Was this translation helpful? Give feedback.
-
|
You can try it, it probably won't work though at this point |
Beta Was this translation helpful? Give feedback.
-
|
I have it working on an nvidia A30. which is the next generation of the turing series. |
Beta Was this translation helpful? Give feedback.
-
|
How to rebuild ? :)
|
Beta Was this translation helpful? Give feedback.
-
|
@yeahme49 would you please post some instructions how to rebuild against compute 7.5 from your github? I tried "docker build" but got too many errors. :( |
Beta Was this translation helpful? Give feedback.
-
|
If you pull the latest image, it should now compile the models at first run if they don't exist so it should work on any system now. A couple things are needed, in docker-compose.yml you need a path for the models so they get retained. model in config.yml should look like or if you want to use the smaller 288 model needs to be in config.yml also that it uses tensorrt Let me know if this works, I tested it on my system and it worked but I only have my one card to test with, but I believe it should work for any nvidia card. |
Beta Was this translation helpful? Give feedback.
-
https://docs.frigate.video/contributing#setup
|
Beta Was this translation helpful? Give feedback.
-
So, yolov4-tiny-416.trt works as expected (can detect all types of objects) with random HQ mp4 file downloaded from the internet but does not (I don't know why it works well with person detection only even if its far and blurry) with my not-so-high-quality HD camera unlike tflite Coral model bundled with frigate. Where can I download or how can I build better model like a non-tiny yolov4 or similar? |
Beta Was this translation helpful? Give feedback.
-
Thanks @yeahme49. Any chance you still have and can share the previous docker image? For the time being I was hoping to roll back to the last working version, but didn't realize it was no longer on dockerhub when upgrading. |
Beta Was this translation helpful? Give feedback.
-
|
I'm pushing a new image to docker hub that will allow disabling FP16 during compiling models, even though mine compiles fine with those errors we'll see if allowing you to disable FP16 will help. If not, I'll upload the compiled models from the old image that work on pascal. |
Beta Was this translation helpful? Give feedback.
-
Oddly, with the newest image even with the default yolov4-tiny-416 and FP16=true it's working for me. Haven't had a chance to play with the other models or see if there's an effect of disabling FP16, it looks like I'm back in business. Thanks! |
Beta Was this translation helpful? Give feedback.
-
|
@yeahme49 any chance to have 0.10-stable image with these features included? |
Beta Was this translation helpful? Give feedback.
-
|
Sorry only 0.11 |
Beta Was this translation helpful? Give feedback.
-
|
Running on unraid from dockers 3060ti and working fantasticly.
love your work.
Get Outlook for Android<https://aka.ms/AAb9ysg>
…________________________________
From: nickp27 ***@***.***>
Sent: Tuesday, November 15, 2022 8:07:31 AM
To: blakeblackshear/frigate ***@***.***>
Cc: IronBeardKnight ***@***.***>; Comment ***@***.***>
Subject: Re: [blakeblackshear/frigate] TensorRT using Nvidia GPU (Discussion #3016)
Has the docker file for the tensorrt build been taken down?
—
Reply to this email directly, view it on GitHub<#3016 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ANU2ZQF5LKUXQGJDKSZ6YI3WIKZ2HANCNFSM5RVRZWOQ>.
You are receiving this because you commented.Message ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
Very cool! As a Docker noob, I was surprised how easy this was to install, just:
I previously had inference times of 100-200ms with 2 CPUs; now it's ~10ms. |
Beta Was this translation helpful? Give feedback.
-
|
Absolutely love this. Thank you so much for giving us this option. My Coral TPUs have been on backorder since April and I just happened across this project while researching GPU decoding for Frigate. I fired up a new VM to test it out and was very impressed, so much so that I promptly shut down the Frigate VM that was using the cpu detectors. I'm currently trying to push Frigate to its limits. I have 15 cameras at 1080p or higher, detect is using the primary camera feed instead of the lower res sub feed. I have a Tesla P4 running at normal clocks for detect and also h264 decoding. The gpu is only at about 15% but the decoder is at 100% with an inference speed of around 5.15 (it was over 100 with cpu detectors even with using sub feeds for detect). A quick question. I have a second Tesla P4 and hopefully some Coral TPUs soon. Can this version of Frigate utilize multiple GPUs or a mix of a GPU and the Coral TPUs for detectors? I've tried defining two GPU detectors using the device flags under each detector (device: pci:0 and device: pci:1) but only one is utilized. Thank you again for all your hard work. |
Beta Was this translation helpful? Give feedback.
-
|
@phabibjr, have you checked to see that both GPUs are being passed to your container with the |
Beta Was this translation helpful? Give feedback.
-
|
Yes. Both gpus show up inside the container. |
Beta Was this translation helpful? Give feedback.
-
|
Compose defaults to using all gpus but I’ve tried it with and without the ‘count: all’ argument under gpu reservation. In both cases NVIDIA-smi shows both cards but only one being utilized. It could be that it only uses one until it reaches high utilization then starts utilizing the next card. I’m going to play around with ffmpeg tonight to see if I can split up the decoding load. |
Beta Was this translation helpful? Give feedback.
-
|
When it comes to ffmpeg, a GPU id has to be specified if multiple are available https://arstech.net/choose-gpu-in-ffmpeg/ |
Beta Was this translation helpful? Give feedback.
-
|
Thank you. I thought I had seen something about that before, I’ll give the article a read tonight and see what kind of results I get with it. |
Beta Was this translation helpful? Give feedback.
-
|
I am sorry if I am asking a question that has been asked before, just want to understand the current status: Is currently a way to run Frigate on Jetson Nano 4GB, using the full potential of NVIDA's 128 cores? If the answer is 'yes' - can I take such a 100% working image from a repository (where?) and run it, so it will work? If the answer is 'not yet', then what is the status of the work done, if any? Please understand - I am asking specifically for the use case with Jetson Nano. I don't have any other GPU/TPU sitting around, but I do have one Jetson... Thanks. |
Beta Was this translation helpful? Give feedback.
-
|
read my post of today :) |
Beta Was this translation helpful? Give feedback.
-
|
First, @yeahme49 thanks for making this available. I am looking forward very much using it, however, I run into a persisting error that i cannot overcome for days now.... frigatetensor | [s6-init] making user provided files available at /var/run/s6/etc...exited 0. the yolo4 directery does get filled up as expected and contains the required files. Any help would be highly appreciated ;-) |
Beta Was this translation helpful? Give feedback.
-
|
I am a Linux noob, but I finally managed to get this working on a Proxmox with in a docker container on a Ubuntu 22.04 LCX with Nvidia 515.86 driver (with CUDA 11.7) for Nvidia Geforce 970 card. I used this guide to install the Nvidia drivers on host and LXC: https://gist.github.com/egg82/90164a31db6b71d36fa4f4056bbee2eb And then this guide to install Nvidia container tooltkit: https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#install-guide Ran into some strange error messages when running the frigatetensor docker. This line fixed it: Also had to define path: |
Beta Was this translation helpful? Give feedback.
-
|
Yea I don’t think the problem is with the GPU at the point. It’s now getting it to run on other platforms such as ARM and such.
Personally I want to run CompreFace on a Jetson Nano and have Frigate on a VMWare Install of Ubuntu Server LTS with both a Nvidia card passed through for transcoding, and a USB Coral to handle the object detection. Then via MQTT pass that info to Double Take on my Home Assistant install which will send it to the Nano for Facial Recognition and Verification
On Jan 28, 2023, at 1:59 PM, opnordahl ***@***.***> wrote:
I am a Linux noob, but I finally managed to get this working on a Proxmox with in a docker container on a Ubuntu 22.04 LCX with Nvidia 515.86 driver (with CUDA 11.7) for Nvidia Geforce 970 card.
I used this guide to install the Nvidia drivers on host and LXC: https://gist.github.com/egg82/90164a31db6b71d36fa4f4056bbee2eb
And then this guide to install Nvidia container tooltkit: https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#install-guide
Ran into some strange error messages when running the frigatetensor docker. This line fixed it:
sudo sed -i 's/^#no-cgroups = false/no-cgroups = true/;' /etc/nvidia-container-runtime/config.toml
Also had to define path:
volumes:
- /path/to/yolo4/directory:/yolo4
—
Reply to this email directly, view it on GitHub<#3016 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACGXNJHVPUN4A6QP4ZBNETLWUV3CPANCNFSM5RVRZWOQ>.
You are receiving this because you commented.Message ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
I deployed yeahme49/frigatetensor:latest on a TrueNAS SCALE box with ~20TB of storage and a Quadro A4000 I have ~8 4K cameras running with the default built model files and killer stability. @yeahme49 - If you are a SF / Seattle adjacent, I owe you beer. I wrote a small how to for getting this container working on SCALE, and posted to /r/homelab and /r/frigate but was promptly ignored for including too many links in my guide :( Question, Are there plans to include changes like .12 where MQTT becomes optional? |
Beta Was this translation helpful? Give feedback.
-
0.12 supports tensorrt directly, is there any reason it is not sufficient as is? |
Beta Was this translation helpful? Give feedback.
-
|
I don't keep entirely up to date on frigate, this is good to know that official .12 supports TensorRT. Next time I don maintenance I will check it out. I have a combo of 3MP and 5MP RTSP cams, so I may be quite close to the memory limits of my personal QuadroP400 card |
Beta Was this translation helpful? Give feedback.
-
|
Would love to see the guide for truenas scale. |
Beta Was this translation helpful? Give feedback.
-
|
Hy all, is there any tutorial to achieve this that can be followed by noobs like me :D .. I nearly broken my HA server 2 times already trying to follow you |
Beta Was this translation helpful? Give feedback.
-
|
This is officially supported in frigate 0.12 which has documentation up |
Beta Was this translation helpful? Give feedback.
-
|
Is there a way to try out a beta or RC?
On Feb 5, 2023, at 2:31 PM, Nicolas Mowen ***@***.***> wrote:
This is officially supported in frigate 0.12 which has documentation up
—
Reply to this email directly, view it on GitHub<#3016 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACGXNJBR34ILUTFRJSIRWF3WWAE2BANCNFSM5RVRZWOQ>.
You are receiving this because you commented.Message ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
@LordNex yes of course https://github.com/blakeblackshear/frigate/releases/tag/v0.12.0-beta7 |
Beta Was this translation helpful? Give feedback.
-
@NickM-27 Wow! |
Beta Was this translation helpful? Give feedback.
-
|
And I’ve gotten a bit closer to testing it out on my Nano. I have it disassembled, now I just gotta flash the card with I’m guessing the standard Nano SDK and start adding dependencies. I’ll let you know how that goes.
I’m also thinking of grabbing one of my old video cards and slapping it into the PowerEdge R620. Since Frigate is already running there fine with the Coral, I’ll probably try and pass it through to my Home Assistant VM where I have CompreFace installed or do a new build of Ubuntu Server and let CompreFace run inside its own VM with the GPU passed through to it.
Then I’ll benchmark what runs Frigate best. The Nano with Coral and CompreFace on the R620. Or ditch the Nano all together and run Frigate in one VM with a Coral, another VM with CompreFace and a GPU passed through, and DoubleTake on Home Assistant to tie everything together.
Lord_Nex
On Feb 5, 2023, at 2:31 PM, Nicolas Mowen ***@***.***> wrote:
This is officially supported in frigate 0.12 which has documentation up
—
Reply to this email directly, view it on GitHub<#3016 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACGXNJBR34ILUTFRJSIRWF3WWAE2BANCNFSM5RVRZWOQ>.
You are receiving this because you commented.Message ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
@NickM-27 When it's trying to connect to MQTT it's now failing. It wasn't previously and the only setting I changes was the image name. I still need to troubleshoot a few things on the MQTT end but like I said. It was connected and working with those same setting under 0.11.1 Also, when I goto Config and attempt to make a change and save I receive "Could not write config file, be sure that Frigate has write permission on the config file." |
Beta Was this translation helpful? Give feedback.
-
@NickM-27 Ok so I went through and added the new settings for birdseye to my config changing only the "mode:" value to "continuous". I've rebooted both HA with the Frigate Proxy Beta and Frigate but do not see any new "birdseye" camera entity such as "camera.frigate_birdseye" or something. If I was to add the RSTP restream directly to the card; where and what would I put in the card entity? I'd prefer to keep using the Frigate Card with its integrations and not resort to adding some other card just to get the RSTP stream. |
Beta Was this translation helpful? Give feedback.
-
I'd suggest reading the docs, because that mode is unrelated to restream https://deploy-preview-4055--frigate-docs.netlify.app/configuration/restream |
Beta Was this translation helpful? Give feedback.
-
|
Also, have you updated to the beta version of the integration? that is required as well |
Beta Was this translation helpful? Give feedback.
-
I believe so. I upgraded it when I went to 0.11.1. Is there a different one that what I'm using below?
|
Beta Was this translation helpful? Give feedback.
-
|
I would highly suggest reading the release notes before updating to a release. If nothing else at least read the breaking changes section which is conveniently at the top of the release notes. The first item listed there:
|
Beta Was this translation helpful? Give feedback.
-
|
folks - very excited to discover frigate/nvidia thread. i started few years ago running deepstack then migrated to torchserve then migrated to nvidia triton inference server, used yolo v4/v5/v7 (soon to migrate to v8) all pushing notifications from 14 cameras to home assistant. few questions:
|
Beta Was this translation helpful? Give feedback.
-
|
@olyashok Any discussions are most welcome ! |
Beta Was this translation helpful? Give feedback.
-
|
That's excellent work with getting frigate running on Nvidia GPUs. Lately I saw that the docker image has been removed from your docker hub repo. Secondly, are there any instructions on how to build the tensorrt docker image which you created ? have been updated. Is it available on your frigate fork? Thank you !! |
Beta Was this translation helpful? Give feedback.
and others
-
I was able to make an Nvidia GPU work for detection by modifying this https://github.com/yury-sannikov/frigate/tree/gstreamer (#2548)
I started by using the watsor dockerfile (https://github.com/asmirnou/watsor/blob/master/docker/Dockerfile.base) but updating it for ubuntu 20.04: https://pastebin.com/ERhsFwEi
and then the watsor gpu dockerfile (https://github.com/asmirnou/watsor/blob/master/docker/Dockerfile.gpu.base) using my new local image: https://pastebin.com/HnBpHWJX
Then I modified the yolo4 converter files for amd64 instead of arm.
Dockerfile: https://pastebin.com/iHbGPR9Z
build.sh: https://pastebin.com/SnA9gibi
assets/install_protobuf.sh: https://pastebin.com/xHfdG3V8
assets/run.sh: https://pastebin.com/XPC0qmCe
Once the models completed, I copied the .trt files from yolo4/model and the .so from yolo4/plugin to a tensor_assets folder i made in my build folder. Then I updated the Dockerfile for amd64 instead of arm: https://pastebin.com/Ey6vFyRh
Once I had that image I was able to have my docker-compose file use that image instead of the standard frigate image. Added
model: path: /yolo4/yolov4-tiny-416.trt labelmap_path: /labelmap.txt width: 416 height: 416to my config.yml file and started it up.Using a 2GB Quadro P400 card my inference speed is around 16-17ms (9-10ms if using yolo4-tiny-288.trt). I run frigate on a Proxmox LXC container with 3 CPU cores assigned and 2GB of RAM, using nvidia hardware decoder. Previously my inference times were between 80-100ms and 75-80% CPU usage on that container.
I'm sure the docker files can be cleaned up and the process streamlined, there is probably some overlap between what gets built and installed between the different docker images since I was combining multiple other docker files.
If anyone is able to test this out and see if it works for them as well that would be awesome. Let me know if anything I posted doesn't work incase I forgot to add any changes i had to make. If others can get it working hopefully it can get cleaned up and added at some point.
Beta Was this translation helpful? Give feedback.
All reactions