[HW Accel Support]: Rockchip ffmpeg / GPU #10383
-
Beta Was this translation helpful? Give feedback.
Replies: 4 comments 1 reply
-
|
@MarcA711 correct me if I am wrong, but I don't think GPU usage statistics were implemented for rockchip? |
Beta Was this translation helpful? Give feedback.
-
|
@NickM-27 No, GPU statistics are not yet implemented. |
Beta Was this translation helpful? Give feedback.
-
|
@csvke you use a luckfox Pico max with an rv1106 SoC, aren't you? As far as I know these Socs can only encode frames, but not decode. Do you have a Camera attached to your board directly using MIPI CSI? Perhaps you can pass the raw frames directly to frigate using input args. Moreover, rknn won't work by default on your board. Maybe I can add support for it, if you want to help me with testing? |
Beta Was this translation helpful? Give feedback.
-
|
I did some research. Your board supports hardware encoding but not hardware decoding. Moreover, there is no FFmpeg fork that supports hardware encoding or reading frames from the MIPI CSI interface currently. Since frigate uses FFmpeg, the only way to get the frames to frigate is what you already do: use the RK MPI SDK to stream the frames from the camera to rtsp and feed this rtsp stream to frigate. Of course this is not very efficient, since FFmpeg needs to decode these rtsp frames in software. The best way using your hardware was to use FFmpeg to read the raw frames from MIPI CSI and pass them directly to frigate and additionally encode these frames using hardware acceleration in order to store them as a recording. This would greatly reduce the CPU usage. Maybe in the future some developer adds support for your soc and it's SDK to FFmpeg. However, if I add some libraries and rknn models, you could use your NPU for object detection. Would this help you? |
Beta Was this translation helpful? Give feedback.
-
|
One additional comment @csvke : If you know some C programming, you should develop your own software. It could work similar to this (simplified) pseudocode: You can connect to rtsp using e.g. VLC Media Player on your phone or computer to see the live stream. And you can share the If you are interested, this wiki covers almost everything you need to know about using MIPI CSI, video encoding and NPU: https://wiki.luckfox.com/Luckfox-Pico/Luckfox-Pico-quick-start/ |
Beta Was this translation helpful? Give feedback.
I did some research. Your board supports hardware encoding but not hardware decoding. Moreover, there is no FFmpeg fork that supports hardware encoding or reading frames from the MIPI CSI interface currently.
Since frigate uses FFmpeg, the only way to get the frames to frigate is what you already do: use the RK MPI SDK to stream the frames from the camera to rtsp and feed this rtsp stream to frigate. Of course this is not very efficient, since FFmpeg needs to decode these rtsp frames in software.
The best way using your hardware was to use FFmpeg to read the raw frames from MIPI CSI and pass them directly to frigate and additionally encode these frames using hardware acceleration in order…