webrtc pointers

[figadore]

If this belongs in the go2rtc repo, let me know

I have a few cameras, like the Reolink RLC-820A and Duo 2. I've spent days trying to get this to work the way I'd like, but still running into issues. I would like to view at least 2 (maybe 3 or 4) "real time" high quality streams on the Frigate "Cameras" page as well as in Home Assistant. It seemed like go2rtc and webrtc were the way to go.

These Reolink cams offers two streams, the "main" 4k h265 stream, and the "sub" h264 stream. My configs are roughly as follows

ui:
  # Set the default live mode for cameras in the UI
  live_mode: webrtc
go2rtc:
  # Enable webrtc
  webrtc:
    candidates:
      - 10.1.2.2:8555
      - stun:8555
  streams:
    # Hopefully transcoded stream for webrtc livestreaming
    front_porch_264: # Reolink Duo 2
      - "rtsp:https://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@10.1.2.11:554/h264Preview_01_main"  #is actually h265, needs to be transcoded
      - "ffmpeg:front_porch_264#video=h264#audio=opus#hardware"
    # Low res substream for detection
    front_porch_sub_264:
      - "ffmpeg:rtsp:https://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@10.1.2.11:554/h264Preview_01_sub"
      - "ffmpeg:front_porch_sub_264#audio=opus#hardware"
    # Original 4k stream for recording
    front_porch_265:
      - "ffmpeg:rtsp:https://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}[email protected]:554/h265Preview_01_main"
      - "ffmpeg:front_porch_265#audio=opus#hardware"

    nursery_264: # Reolink 820A
      - "rtsp:https://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@10.1.2.12:554/h264Preview_01_main"
      - "ffmpeg:front_porch_264#video=h264#audio=opus#hardware"
    nursery_sub_264:
      - "ffmpeg:rtsp:https://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@10.1.2.12:554/h264Preview_01_sub"
      - "ffmpeg:nursery_sub_264#audio=opus#hardware"
    nursery_265:
      - "ffmpeg:rtsp:https://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@10.1.2.12:554/h265Preview_01_main"

ffmpeg:
  output_args:
    record: preset-record-generic-audio-aac
  hwaccel_args: preset-nvidia-h264

cameras:
  front-porch:
    live:
      stream_name: front_porch_264
    ffmpeg:
      inputs:
        - path: rtsp:https://127.0.0.1:8554/front_porch_sub_264?video=copy&audio=opus
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp:https://127.0.0.1:8554/front_porch_265?video=copy&audio=copy
          input_args: preset-rtsp-restream
          roles:
            - record
      hwaccel_args: preset-nvidia-h264
    record:
      enabled: true
    detect:
      enabled: false # <---- disable detection for now
      width: 1536
      height: 576

  nursery:
    live:
      stream_name: nursery_264
    ffmpeg:
      inputs:
        - path: rtsp:https://127.0.0.1:8554/nursery_sub_264?video=copy&audio=opus
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp:https://127.0.0.1:8554/nursery_265?video=copy&audio=copy
          input_args: preset-rtsp-restream
          roles:
            - record
      hwaccel_args: preset-nvidia-h264
    record:
      enabled: true
    detect:
      enabled: false

I've tried countless variations on the ffmpeg params (mostly because I have no idea what I'm doing), but it should theoretically be possible to transcode the h265 stream as h264, add opus audio, and use that for the high quality livestreaming with webrtc, right? I've had a couple occasions where webrtc works for one camera type for the live view in Frigate, but I still get laggy MSE or worse in Home Assistant. Sometimes I get webrtc to work, but it's just a green screen. Sometimes I just get go2rtc/ffmpeg errors. What are the recommended settings to do what I'm trying to accomplish (assuming it's even possible). I'd be totally fine having a 1k stream too (tips welcome), if that helps solve my problems, it's just that the substream is so low quality that I'd like something better.

So, at a high level, am I even on the right track? go2rtc and ffmpeg should be able to turn a h265 stream into a h264 stream, right?

I can include my go2rtc logs or info/links pages if they would be useful.

[Bettman66]

I've also asked in other threads whether they have to transcode the stream at all.
Edge, Chrome and Safari display h265 with the correct settings.

[NickM-27]

That's incorrect for WebRTC, the only browser that supports h265 WebRTC is safari in experimental mode

[Bettman66]

He said webrtc was the right way not that he had to use webrtc.

I posted a config for h265 here.
Works for Chrome, Edge and Safari.

[figadore]

Created an issue at #9469, which has made a good amount of progress (but still has unresolved questions at the time of writing this)