Replies: 14 comments 28 replies
-
I am currently using a N4100 soft router to run Frigate. When there is no one around, the page shows an inference speed of 20-30 milliseconds, but when there are people, it becomes 70-80 milliseconds. The actual feeling is that it takes two to three seconds for Home Assistant to get the information, which is too slow. I plan to purchase an Orange Pi 5, so I am particularly interested in the following two points: 1.Is an Orange Pi 5 with 4GB of memory enough to run Frigate? |
Beta Was this translation helpful? Give feedback.
-
Are these models really 320x320? I thought the YOLO NAS models were 640x640. |
Beta Was this translation helpful? Give feedback.
-
Frigate version: 0.14.0-b97e274 |
Beta Was this translation helpful? Give feedback.
-
Frigate version: 0.14.0-b97e274 |
Beta Was this translation helpful? Give feedback.
-
Frigate version: frigate:dev-d646338-rk |
Beta Was this translation helpful? Give feedback.
-
I have tried all deci-fp16-yolonas_* models. Trying to move from coral-based system. All of these models seem to weigh heavily on "cat". Same config (except for model and detectors). In 0.13.2 coral system, dogs were dogs, and people were people. Now dogs are always cats and people are sometimes cats. Rarely are there any actual cats. Are there any other models to try with the rockchip decoder? Should I just remove cat from object detection and see if that improves classification? |
Beta Was this translation helpful? Give feedback.
-
With the known issue of 8GB and 16GB rockchip boards having crashes, isn't this really a non-starter for frigate until this is fixed? |
Beta Was this translation helpful? Give feedback.
-
With the beta v3 image, when restreaming 1024x768 mjpg stream from an esp32 camera module to m264 I'm getting about I'm 20% cpu usage on my rk3588. Feels like go2rtc ffmpeg is not using hardware acceleration yet. |
Beta Was this translation helpful? Give feedback.
-
I'm not sure if anyone knows but should the rknn detector just keep incrementally increase memory usage? It seems to go from 1% to about 8% usage after a day and then will flush back to 4% usage. Is this set somewhere in frigate? If I use a coral usb it stays around 2% usage so I wasn't sure if there was a reason it spikes so much higher vs the coral usb. |
Beta Was this translation helpful? Give feedback.
-
Frigate version: 0.13.2-69d9a261 |
Beta Was this translation helpful? Give feedback.
-
Frigate version: 0.13.2-69d9a261 |
Beta Was this translation helpful? Give feedback.
-
Frigate version: 0.14.0-da913d8 |
Beta Was this translation helpful? Give feedback.
-
It seems that there is an incompatibility between the yolov8 models from Frigate v13 and newer kernels. This results in this error:
I recommend upgrading to Frigate v14 since yolov8 is not supported anymore. If you encounter this problem in Frigate v14, please reply to this comment. This issue was initially reported here: nyanmisaka/ffmpeg-rockchip#95 |
Beta Was this translation helpful? Give feedback.
-
Hi @MarcA711 , @ALL, FYI: I tried the 14.0 release on the 6.1 image from here. Linux ubuntu 6.1.0-1021-rockchip #21-Ubuntu SMP Mon Jul 29 03:52:32 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux Good news - inference is working on one camera with good speed. I did not try with more cameras yet, this is planned. ![]() Frigate version: 0.14.0-da913d8 Still I was expecting better detector performance. My other setup is running on the pretty old Intel Core i7-6700K and inference there on ov detector is 10-20 ms. Given that the Rk3588 has the hardware acceelerator with 3 NPU cores and also hardware decoding of h.265 stream, I was expecting it to be at least on par with the old Core i7. Many thanks for your help and advice! |
Beta Was this translation helpful? Give feedback.
-
This is intended as an exchange for Rockchip users. We can talk about problems setting up hardware acceleration, optimizing performance, which distribution works best etc. Moreover, I want to collect some information like which OS + board works or not as well as inference times on different SoCs. I hope that as many users as possible help and submit data. Finally, I would also like to compile a list of FAQ, common pitfalls and known issues.
Latest news
The current beta version of Frigate includes lots of improvements in both hardware video processing and object detection for Rockchip devices. I would appreciate any help testing these changes. However, note that this is a beta version, so use with caution.
In this version, the option
core_mask
of the rknn detector was replaced bynum_cores
. You don't pass a complicated bit mask any more but just the number of cores you want to use (e.g.num_cores: 3
for rk3588 SoCs).Moreover, Frigate Rockchip uses yolo-nas instead of yolov8. Yolo-nas is considered superior to yolov8. It is not suggested to stick to yolov8, since it is not supported anymore. Just changing the
model: path:
to a yolov8 model will not work because Frigate does not implement post-processing for yolov8. However, keep in mind that yolo-nas is not suited for commercial usage, see this license.There was a kernel bug in version 6.1.0-1018-rockchip and 6.1.0-1019-rockchip of
ubuntu-rockchip
's kernel. Upgrade to 6.1.0-1020-rockchip or above. To check your version, type:Docker image: ghcr.io/blakeblackshear/frigate:0.14.0-rc2-rk
Beta docs: https://deploy-preview-11419--frigate-docs.netlify.app/
Also see these notes for the beta release.
System compatibility
You can help complete this table by commenting these information:
Inference times
All times are in ms.
You can help complete this table by commenting these information:
FAQ
Is there a Frigate add-on for Home Assistant with support for Rockchip hardware?
No. There are (at least) 3 ways to install Home Assistant: their OS (HAOS), their supervised script (HA Supervised) and their docker image (HA Container). Only HAOS and HA Supervised support add-ons (see this comparison for more details).
HAOS uses the mainline Linux kernel that currently lacks the necessary drivers for Rockchip hardware acceleration (see this table to see the progress of mainlining the rk3588). HA Supervised currently lacks the option to unmask paths (
--security-opt systempaths=unconfined
, see home-assistant/supervisor#4863). So there will be no add-on for Rockchip hardware until either HAOS or HA Supervised works.The only way to install Home Assistant and Frigate with Rockchip support is to use HA Container. For details see the next FAQ "How do I install Frigate and Home Assistant?". Also note that you don't need Home Assistant to use Frigate. Frigate already has a UI that received a major overhaul in v0.14. Using Home Assistant with Frigate makes only sense if you already use HA for other services and prefer an all-in-one solution or if you want to trigger other actions for certain events.
How do I install Frigate and Home Assistant?
This is currently only possible using HA Container (see previous FAQ "Is there a Frigate add-on for Home Assistant with support for Rockchip hardware?" for details). Note that you can't install add-ons in HA Container (see this comparison for more details). However, most add-ons are also available as docker images, you just need to configure them manually.
Please note:
The instructions below assume, that you are in an empty directory with read and write permission.
Now, create the files
docker-compose.yml
andmosquitto-data/config/mosquitto.conf
and paste the contents below. Thedocker-compose.yml
creates a docker network and adds all containers to it. This way the containers can communicate with each other. Instead of an IP address you can use thecontainer_name
of each container.e.g.
nano docker-compose.yml
e.g.
nano mosquitto-data/config/mosquitto.conf
Now start just the mosquitto container and add two users for Frigate and Home Assistant. Remember the password that you choose. Afterwards, create the
frigate-data/config/config.yml
. I highlighted some lines that you might need to change.e.g.
nano frigate-data/config/config.yml
Now start Home Assistant and install HACS, afterwards start all containers:
Finally, you can open Home Assistant in your browser using
http:https://IP-of-your-server:8123
. Go toSettings --> Integrations --> Add integration
. There you can setup HACS, afterwards MQTT, afterwards Frigate (download it from HACS first). After installing HACS you should restart Home Assistant usingdocker compose restart
in your terminal. During the setup of MQTT you are asked to enter the broker addressmosquitto
as well as the usernamehomeassistant
and password that you chose earlier. During the Frigate setup you need to enter the addresshttp:https://frigate:5000
.You can stop your setup using
docker compose down
, start usingdocker compose up -d
and restart usingdocker compose restart
if you are in the directory where you set everything up. Now, you should study the docs of each project and adopt everything to your needs.Known issues
Streams crash or recordings are missing
This is due to a hardware limitation that prevents the video processing hardware to access memory outside the first 4GB. This is potentially fixable, see rockchip-linux/mpp#560.
Beta Was this translation helpful? Give feedback.
All reactions