Reduce latency on older hardware ( Hardware Acceleration not supported )

[MatthewReed303]

Hi, I have 4x cheap IP cameras I will be mounting on some machinery for operator to help see and align logs on a saw mill to the laser line, The operators station is remote and no one is close to the saw, it's just for alignment purposes. The issue I'm having is the latency is 1-2 sec in frigate, I did get it down from around 4-5 sec factory settings. Here is a link to the cam:

https://www.aliexpress.us/item/3256803348748715.html?spm=a2g0o.order_list.order_list_main.11.44ab1802nCqUaz&gatewayAdapt=glo2usa4itemAdapt

It supports all the normal features such as rstp, onvif etc it is using H265 and not H264 so not sure if that's a cause or not. If I run there xmeye VMS software in a windows VM the latency it almost nothing <1 sec . I have tried the stream in VLC with low latency settings and also OpenCV directly and these have around the same latency as Frigate at 1-2sec which makes me believe this is an RTSP limitation or an issue with FFMPEG?

Here is my Frigate Config.

mqtt:
  enabled: False

go2rtc:
  streams:
    rtsp_cam51:
      - rtsp:https://:@192.168.1.51:554/user=admin_password=tlJwpbo6_channel=1_stream=0&amp;onvif=0.sdp?real_st # <- stream which supports video & aac audio.
      #- "ffmpeg:rtsp_cam#video=h264"
      #- "ffmpeg:rtsp_cam#audio=opus" # <- copy of the stream which transcodes audio to opus

    rtsp_cam52:
      - rtsp:https://:@192.168.1.52:554/user=admin_password=tlJwpbo6_channel=1_stream=0&amp;onvif=0.sdp?real_st # <- stream which supports video & aac audio.
      #- "ffmpeg:rtsp_cam#video=h264"
      #- "ffmpeg:rtsp_cam#audio=opus" # <- copy of the stream which transcodes audio to opus

    rtsp_cam53:
      - rtsp:https://:@192.168.1.53:554/user=admin_password=tlJwpbo6_channel=1_stream=0&amp;onvif=0.sdp?real_st # <- stream which supports video & aac audio.
      #- "ffmpeg:rtsp_cam#video=h264"
      #- "ffmpeg:rtsp_cam#audio=opus" # <- copy of the stream which transcodes audio to opus

cameras:
  cam51:
    ffmpeg:
      input_args: -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp:https://127.0.0.1:8554/rtsp_cam51 # <--- the name here must match the name of the camera in restream
          input_args: preset-rtsp-restream
          roles:
            - detect
#      hwaccel_args: preset-vaapi
#    live:
#      stream_name: rtsp_cam
    detect:
      enabled: False # <---- disable detection until you have a working camera feed
      width: 1280 # <---- update for your camera's resolution
      height: 720 # <---- update for your camera's resolution
      fps: 10

  cam52:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp:https://127.0.0.1:8554/rtsp_cam52 # <--- the name here must match the name of the camera in restream
          input_args: preset-rtsp-restream
          roles:
            - detect
#      hwaccel_args: preset-vaapi
#    live:
#      stream_name: rtsp_cam
    detect:
      enabled: False # <---- disable detection until you have a working camera feed
      width: 1280 # <---- update for your camera's resolution
      height: 720 # <---- update for your camera's resolution
      fps: 10

  cam53:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp:https://127.0.0.1:8554/rtsp_cam53 # <--- the name here must match the name of the camera in restream
          input_args: preset-rtsp-restream
          roles:
            - detect
#      hwaccel_args: preset-vaapi
#    live:
#      stream_name: rtsp_cam
    detect:
      enabled: False # <---- disable detection until you have a working camera feed
      width: 1280 # <---- update for your camera's resolution
      height: 720 # <---- update for your camera's resolution
      fps: 10

# Include all cameras by default in Birdseye view
birdseye:
  enabled: True
  mode: continuous

# Optional
ui:
  # Optional: Set the default live mode for cameras in the UI (default: shown below)
  live_mode: mse
  # Optional: Set a timezone to use in the UI (default: use browser local time)
  # timezone: America/Denver
  # Optional: Use an experimental recordings / camera view UI (default: shown below)
  use_experimental: True
  # Optional: Set the time format used.
  # Options are browser, 12hour, or 24hour (default: shown below)
  time_format: browser
  # Optional: Set the date style for a specified length.
  # Options are: full, long, medium, short
  # Examples:
  #    short: 2/11/23
  #    medium: Feb 11, 2023
  #    full: Saturday, February 11, 2023
  # (default: shown below).
  date_style: short
  # Optional: Set the time style for a specified length.
  # Options are: full, long, medium, short
  # Examples:
  #    short: 8:14 PM
  #    medium: 8:15:22 PM
  #    full: 8:15:22 PM Mountain Standard Time
  # (default: shown below).
  time_style: medium
  # Optional: Ability to manually override the date / time styling to use strftime format
  # https://www.gnu.org/software/libc/manual/html_node/Formatting-Calendar-Time.html 

[NickM-27]

Where are you measuring this latency? Detect feed or live view in Frigate?

[Teklock]

go2rtc doesn't offer the ability to view multiple cams in one page like the "Birdseye" view. I know you can view the birdseye view through go2rtc if you have restreaming turned on, but the actual camera feeds are still the detection feed. I'm with the OP, it would be nice to specify what feed the birdseye view uses. I'm still new to this so let me know if I'm missing something.

[NickM-27]

go2rtc doesn't offer the ability to view multiple cams in one page like the "Birdseye" view.

it does offer this, which is why I suggested it

would be nice to specify what feed the birdseye view uses. I'm still new to this so let me know if I'm missing something.

You are missing something. As I said, birdseye requires the streams to be decoded since all the frames from separate cameras are composited into a new stream. This takes latency and there is no way around that. If you want a single page with multiple camera views that have their own stream then go2rtc offers this exact thing

[Teklock]

Is it possible to configure the birdseye view to use something other than the detection feed?

[NickM-27]

Is it possible to configure the birdseye view to use something other than the detection feed?

no, and it would be irrelevant because whichever feed birdseye uses would have to be decoded which introduces latency so the latency would still not be realtime.

[MatthewReed303]

Hi @Teklock I ended up building my own app with https://github.com/frangoteam/FUXA which is an industrial HMI/SCADA web software. I used Iframes and the go2rtc RTC url stream in the iframe, works well and only 500ms - 1sec latency compared to the 2-3sec with the birdseye view. I can share you my project if you are interested

[Teklock]

@MatthewReed303 Thank you! You are very kind, but I was piggybacking on your birdseye request more for framerate and resolution reasoning. Although, I thought it would be possible to create the birdseye view within your 1 sec latency requirement if the feeds were not also being used for detection. Apparently, that's not the case.