Add feature/option to rotate cameras #5653

horttorrell32 · 2023-03-06T13:50:18Z

Added field in the configuration file to rotate camera: 0 (default), 90, 180 or 270 (-90º) degrees.
Enable this feature to detect and record output.
Update the documentation.
Add new test to test_ffmpeg_presets.

…32/frigate into feature/RotateCamera

netlify · 2023-03-06T13:50:25Z

✅ Deploy Preview for frigate-docs ready!

Name	Link
🔨 Latest commit	`21c4135`
🔍 Latest deploy log	https://app.netlify.com/sites/frigate-docs/deploys/640c858411941700070c0628
😎 Deploy Preview	https://deploy-preview-5653--frigate-docs.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

NickM-27 · 2023-03-06T13:54:14Z

frigate/ffmpeg_presets.py

@@ -52,6 +52,17 @@ def get_selected_gpu(self) -> str:
 f"FFmpeg Frigate/{VERSION}",
 ]

+PRESETS_FFMPEG_HW_ACCEL = {


What's the reason for breaking this out? The encoding still has different set of args for each preset anyway

NickM-27 · 2023-03-06T14:00:04Z

frigate/ffmpeg_presets.py

 scale = PRESETS_HW_ACCEL_SCALE.get(arg, "")

 if scale:
- scale = scale.format(fps, width, height).split(" ")
+ scale = scale.format(fps, width, height, transpose).split(" ")


Seems like this is going to lead to multiple -vf args, I think this logic needs to be refactored.

We're also going to need more logic for the vf formatting as many of these are going to need hwupload/hwdownload args

This is still an issue I think, the scale and rotation both have -vf which will cause issues

NickM-27 · 2023-03-06T14:00:36Z

docs/docs/configuration/cameras.md

+ roles:
+ - detect
+ - record
+ rotate: 90


I think this would be better as a field of the ffmpeg config and not the camera

NickM-27 · 2023-03-06T14:03:53Z

frigate/test/test_ffmpeg_presets.py

@@ -156,6 +291,142 @@ def test_ffmpeg_output_record_not_preset(self):
 " ".join(frigate_config.cameras["back"].ffmpeg_cmds[0]["cmd"])
 )

+ def test_ffmpeg_output_record_rotate_90_preset(self):


Have these all been tested? (If not, which ones?)

At least some of these I believe wouldn't actually work in ffmpeg because they're missing the hwupload/hwdownload and formatting args.

I've spent a lot of time getting these to manually work with rotation with my nvidia and vaapi setups so definitely need to have the ffmpeg commands for the test confirmed working.

NickM-27 · 2023-03-06T14:06:12Z

frigate/ffmpeg_presets.py

+ else : # Rotation not need or not supported
+ return ""
+
+ return PRESETS_HW_ACCEL_SCALE_ROTATION.get(arg, "").get(mode).format(transpose)


How does this work with the second .get call? This would fail if calling .get on an empty string (which is the default value for no preset)

NickM-27 · 2023-03-06T14:10:44Z

I'm also curious about the utility of this. I think it would be much preferred (and recommended) to use go2rtc to rotate the stream as a restream so the rotation is only done once instead of twice (detect and record)

NickM-27 · 2023-03-06T14:34:20Z

For example, here is a working ffmpeg command to do rotation for nvidia GPU

ffmpeg -hide_banner -threads 1 -hwaccel cuda -hwaccel_output_format cuda -extra_hw_frames 2 -fflags nobuffer -flags low_delay -timeout 5000000 -user_agent go2rtc/ffmpeg -noautorotate -rtsp_transport tcp -i {input} -r 10 -c:v h264_nvenc -g 50 -profile:v high -level:v auto -preset:v p2 -tune:v ll -vf hwdownload,format=nv12,transpose=1,hwupload -c:a copy -threads 1 -user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp {output}

horttorrell32 · 2023-03-07T07:50:54Z

I am totally agree with you that more efficient way would be to use go2rtc to rotate the stream, but I read that go2rtc doesn't work with cameras with some errors in the input.
The idea is leave to 'configurator' the option to use the rotation in the frigate or go2rtc.
Another question that I have is that go2rtc seems to do the rotation using SW and not HW accelerator.

horttorrell32 · 2023-03-07T08:01:35Z

I am not an expert with ffmpeg, but the ffmpeg command to do rotation for nvidia GPU seems to be executed by CPU and not by GPU.
doesn't NVIDIA offer the GPU rotation?
I have read that exist the 'transpose_npp' HW filter, but like I don't have NVIDIA GPU, I can't test.

NickM-27 · 2023-03-07T11:39:15Z

I am not an expert with ffmpeg, but the ffmpeg command to do rotation for nvidia GPU seems to be executed by CPU and not by GPU.
doesn't NVIDIA offer the GPU rotation?
I have read that exist the 'transpose_npp' HW filter, but like I don't have NVIDIA GPU, I can't test.

Right, but scale_cuda is done on the GPU meaning that filters need to be used to move between GPU and CPU

and npp support can't be built into a distributable ffmpeg build so it's not feasible for us to use it

NickM-27 · 2023-03-07T11:42:34Z

I am totally agree with you that more efficient way would be to use go2rtc to rotate the stream, but I read that go2rtc doesn't work with cameras with some errors in the input.

That's if using a camera directly. Go2rtc recommends using ffmpeg for cameras with bad data which solves that issue

The idea is leave to 'configurator' the option to use the rotation in the frigate or go2rtc.

Makes sense, but the docs should definitely recommend users to use go2rtc

Another question that I have is that go2rtc seems to do the rotation using SW and not HW accelerator.

Right, hardware support is early for go2rtc.

horttorrell32 · 2023-03-07T15:20:10Z

Documentation updated to recommend the usage of go2rtc.
Rotate field moved to camera/ffmpeg configuration.
Add ffmpeg flags to allow rotation with cuda (sw ratation)

horttorrell32 · 2023-03-10T16:51:08Z

Hello NickM-27, I don't have clear if I need to correct something more.
Could you guide me ?

blakeblackshear · 2023-03-11T13:47:49Z

@NickM-27 is unavailable, so this will have to wait until he is back a week from now.

horttorrell32 · 2023-03-11T15:14:40Z

Ok, thanks Blakeblackshear

NickM-27 · 2023-03-18T15:50:40Z

frigate/ffmpeg_presets.py

 scale = PRESETS_HW_ACCEL_SCALE.get(arg, "")

 if scale:
- scale = scale.format(fps, width, height).split(" ")
+ scale = scale.format(fps, width, height, transpose).split(" ")


This is still an issue I think, the scale and rotation both have -vf which will cause issues

NickM-27 · 2023-03-18T15:51:40Z

frigate/ffmpeg_presets.py

+ "default": "-r {0}{3} -s {1}x{2}",
+}
+
+PRESETS_HW_ACCEL_SCALE_ROTATION = {


This isn't doing any scaling just rotation so the name should reflect that

NickM-27 · 2023-03-18T15:53:56Z

frigate/ffmpeg_presets.py

Overall, I think this way of doing it is a bit confusing (to me at least). I think I would rather see a separate dict for DECODE_SCALE and DECODE_SCALE_ROTATE so they are not dependent on each other. Similarly we can have another ENCODE_ROTATE or something so we don't need to worry about splitting a multi layer dict with multiple modes

KenwoodFox · 2023-11-07T04:57:35Z

Did this get merged? Is there a way to rotate the camera views? I don't see anything in the docs...

Galindo, Alex and others added 7 commits March 6, 2023 10:21

Add rotate camera feature to detect option

fd941cd

Add rotate camera feature to record option

826671c

Add rotate camera feature to record option

ead1c36

Merge branch 'blakeblackshear:dev' into feature/RotateCamera

80a9bef

Add hw acceleration capacity to record output.

4049316

Merge branch 'feature/RotateCamera' of https://github.com/horttorrell…

10f4893

…32/frigate into feature/RotateCamera

Added documentation

4299f30

NickM-27 requested changes Mar 6, 2023

View reviewed changes

Correct HW accelerator usage for record output

b14da77

Correct format

50349b2

Galindo, Alex added 3 commits March 7, 2023 15:10

Change the rotate field to camera/ffmpeg

5f51319

Updated doc with recomendation of go2rtc using

f306fe0

Add ffmpeg flag to allow Nvidia roation

082e104

horttorrell32 requested a review from NickM-27 March 7, 2023 15:22

NickM-27 enabled auto-merge (squash) March 7, 2023 15:23

Give the default value for no preset found.

d90594d

auto-merge was automatically disabled March 8, 2023 08:12
Head branch was pushed to by a user without write access

typo

21c4135

NickM-27 requested changes Mar 18, 2023

View reviewed changes

github-actions bot added the stale label Apr 18, 2023

github-actions bot closed this Apr 21, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add feature/option to rotate cameras #5653

Add feature/option to rotate cameras #5653

horttorrell32 commented Mar 6, 2023

netlify bot commented Mar 6, 2023 •

edited

Loading

NickM-27 Mar 6, 2023

NickM-27 Mar 6, 2023

NickM-27 Mar 18, 2023

NickM-27 Mar 6, 2023

NickM-27 Mar 6, 2023 •

edited

Loading

NickM-27 Mar 6, 2023

NickM-27 commented Mar 6, 2023

NickM-27 commented Mar 6, 2023

horttorrell32 commented Mar 7, 2023

horttorrell32 commented Mar 7, 2023

NickM-27 commented Mar 7, 2023

NickM-27 commented Mar 7, 2023

horttorrell32 commented Mar 7, 2023

horttorrell32 commented Mar 10, 2023

blakeblackshear commented Mar 11, 2023

horttorrell32 commented Mar 11, 2023

NickM-27 Mar 18, 2023

NickM-27 Mar 18, 2023

NickM-27 Mar 18, 2023 •

edited

Loading

KenwoodFox commented Nov 7, 2023

Add feature/option to rotate cameras #5653

Add feature/option to rotate cameras #5653

Conversation

horttorrell32 commented Mar 6, 2023

netlify bot commented Mar 6, 2023 • edited Loading

✅ Deploy Preview for frigate-docs ready!

NickM-27 Mar 6, 2023

Choose a reason for hiding this comment

NickM-27 Mar 6, 2023

Choose a reason for hiding this comment

NickM-27 Mar 18, 2023

Choose a reason for hiding this comment

NickM-27 Mar 6, 2023

Choose a reason for hiding this comment

NickM-27 Mar 6, 2023 • edited Loading

Choose a reason for hiding this comment

NickM-27 Mar 6, 2023

Choose a reason for hiding this comment

NickM-27 commented Mar 6, 2023

NickM-27 commented Mar 6, 2023

horttorrell32 commented Mar 7, 2023

horttorrell32 commented Mar 7, 2023

NickM-27 commented Mar 7, 2023

NickM-27 commented Mar 7, 2023

horttorrell32 commented Mar 7, 2023

horttorrell32 commented Mar 10, 2023

blakeblackshear commented Mar 11, 2023

horttorrell32 commented Mar 11, 2023

NickM-27 Mar 18, 2023

Choose a reason for hiding this comment

NickM-27 Mar 18, 2023

Choose a reason for hiding this comment

NickM-27 Mar 18, 2023 • edited Loading

Choose a reason for hiding this comment

KenwoodFox commented Nov 7, 2023

netlify bot commented Mar 6, 2023 •

edited

Loading

NickM-27 Mar 6, 2023 •

edited

Loading

NickM-27 Mar 18, 2023 •

edited

Loading