[Camera Support]: Help to define detect resolutions, Foscam R5, Reolink E1 Pro, Reolink Doorbell

[Chaoscontrol]

Describe the problem you are having

I've been using Frigate for months, but I never used detection till recently when I learned about OpenVINO. I read the docs about detect resolution, but I don't know how to define the right one for my cameras. I'll outline here the available settings, and would like someone just to tell me which resolution would be the right one.

Running all 5 cameras with a single stream, for detection, I avg ~50-60% CPU. Intel N5095, SSD 256GB, 8GB RAM.

Available settings:

Foscam R5 substream

Res: 720p, 640x480, 640x360
Bitrate: 20K - 512K
Frame rate: 1-15
Key frame interval: 10-100
Rate control mode: CBR, VBR

Main stream set at 5MP, 6M bitrate, 25fps, 100 kfi, VBR.

Reolink E1 Pro

Res: 640x360
Bitrate: 64K - 512K
Frame rate: 4, 7, 10, 15

Main stream set at 1440p, 25fps, 5120K

Reolink Doorbell WiFi

Res: 640x480
Bitrate: 64K - 512K
Frame rate: 4, 7, 10, 15
i-frame interval: 1x - 4x

Main stream set at 1920p, 20fps, 6144K, 2x

Template for answers

Res:
Bitrate:
Frame rate:
Key/i-frame interval:
Rate control mode:

AFAIK the FPS should be around 5, and then the detect resolution set in Frigate config to match whatever sub-stream I have, so it doesn't have to be resized and waste resources.

Many thanks

Version

0.14.0-bccffe6

Frigate config file

mqtt:
  host: core-mosquitto
  user: frigate
  password: redacted

# logger:
#   logs:
#     frigate.record.util: debug

go2rtc:
  log:
    #level: trace
    #api: trace
    #exec: trace
    #ngrok: trace
    #rtsp: trace
    #streams: trace
    #webrtc: trace

  # AAC Audio required for recordings, Opus required for webRTC Live View
  streams:
    doorbell: # Reolink Doorbell WiFi
      - rtsp:https://admin:[email protected]:554/h264Preview_01_main
      - ffmpeg:doorbell#audio=aac#audio=volume
      - ffmpeg:doorbell#audio=opus#audio=volume
    reolink: # Reolink E1 Pro broken night vision
      - rtsp:https://admin:[email protected]:554/h264Preview_01_main
      - ffmpeg:reolink#audio=aac#audio=volume
      - ffmpeg:reolink#audio=opus#audio=volume
    bedroom: # Reolink E1 Pro
      - rtsp:https://admin:[email protected]:554/h264Preview_01_main
      - ffmpeg:bedroom#audio=aac#audio=volume
      - ffmpeg:bedroom#audio=opus#audio=volume
    backgarden: # Foscam R5
      - ffmpeg:rtsp:https://foscamadmin:[email protected]:88/videoMain
      - ffmpeg:backgarden#audio=aac#audio=volume
      - ffmpeg:backgarden#audio=opus#audio=volume
    livingroom: # Foscam R5
      - ffmpeg:rtsp:https://foscamadmin:[email protected]:88/videoMain
      - ffmpeg:livingroom#audio=aac#audio=volume
      - ffmpeg:livingroom#audio=opus#audio=volume

  ## Frigate docs recommend HTTP+FLV connection for Reolink Doorbell, but only RTSP allows for 2-way audio
  ## #video=copy is needed for a single line ffmpeg config when using audio tags, or else video stream is dropped


## THIS INCREASES VOL AND PREVENTS THE AUDIO COMING OUT FROM THE CAMERA SOUNDING MUFFLED
## Source https://www.youtube.com/watch?v=upXyBVMR4RM

  ffmpeg:
    bin: ffmpeg
    volume: -af "volume=30dB"

cameras:
  doorbell:
    # enabled: false
    ffmpeg:
      inputs:
        - path: rtsp:https://127.0.0.1:8554/doorbell
          roles:
            - record
            - detect
            - audio
    audio:
      enabled: true
    #birdseye:
    #  order: 1
    zones:
      Front_garden:
        coordinates: 
          0.143,0.527,0.143,0.561,0.373,0.563,0.613,0.56,0.609,0.523,0.885,0.557,0.874,0.704,0.993,0.751,0.995,0.999,0.002,0.999,0,0.557
      Driveway/Neighbours:
        coordinates: 
          0.851,0.516,0.898,0.502,1,0.502,1,0.588,0.973,0.732,0.876,0.703,0.887,0.555,0.834,0.545
    review:
      detections:
        required_zones:
          - Front_garden
          - Driveway/Neighbours
    motion:
      mask: 
        0,0.429,0.122,0.421,0.198,0.33,0.56,0.331,0.589,0.434,0.871,0.412,0.896,0.389,0.975,0.436,0.999,0.436,0.999,0.003,0.002,0
  bedroom:
    # enabled: false
    ffmpeg:
      inputs:
        - path: rtsp:https://127.0.0.1:8554/bedroom
          roles:
            - record
            - detect
            - audio
    audio:
      enabled: true
    #birdseye:
    #  order: 2
    onvif:
      host: 192.168.1.216
      port: 8000
      user: admin
      password: redacted
  reolink:
    # enabled: false
    ffmpeg:
      inputs:
        - path: rtsp:https://127.0.0.1:8554/reolink
          roles:
            - record
            - detect
            - audio
    #birdseye:
    #  order: 2
    onvif:
      host: 192.168.1.138
      port: 8000
      user: admin
      password: redacted
  backgarden:
    # enabled: false
    ffmpeg:
      inputs:
        - path: rtsp:https://127.0.0.1:8554/backgarden
          roles:
            - record
            - detect
            - audio
    #birdseye:
    #  order: 3
    # onvif:
    #   host: 192.168.1.177
    #   port: 888
    #   user: foscamadmin
    #   password: 8LqM7qBX2AX^
    motion:
      mask:
        - 0,0,0,0.471,0.247,0.445,0.275,0.492,0.39,0.492,0.441,0.492,0.56,0.459,0.677,0.439,0.786,0.309,0.768,0.113,0.761,0.003
        - 0.64,0.373,0.578,0.579,0.688,0.708,0.768,0.719,0.803,0.697,0.792,0.292
  livingroom:
    # enabled: false
    ffmpeg:
      inputs:
        - path: rtsp:https://127.0.0.1:8554/livingroom
          roles:
            - record
            - detect
            - audio
    #birdseye:
    #  order: 4
    onvif:
      host: 192.168.1.239
      port: 888
      user: foscamadmin
      password: redacted

## Intel N5095, SSD 256GB, 8GB RAM

ffmpeg:
  hwaccel_args: preset-vaapi
  input_args: preset-rtsp-restream-low-latency
  output_args:
    record: preset-record-generic-audio-aac

detectors:
  ov:
    type: openvino
    device: AUTO

model:
  width: 300
  height: 300
  input_tensor: nhwc
  input_pixel_format: bgr
  labelmap_path: /openvino-model/coco_91cl_bkgr.txt
  path: /openvino-model/ssdlite_mobilenet_v2.xml

record:
  enabled: true
  # sync_recordings: true
  retain:
    days: 1 #default=0
    mode: motion #default
  events:
    retain:
      default: 5
      mode: motion
  export:
    # Optional: Timelapse Output Args (default: shown below).
    # NOTE: The default args are set to fit 24 hours of recording into 1 hour playback.
    # See https://stackoverflow.com/a/58268695 for more info on how these args work.
    # As an example: if you wanted to go from 24 hours to 30 minutes that would be going
    # from 86400 seconds to 1800 seconds which would be 1800 / 86400 = 0.02.
    # The -r (framerate) dictates how smooth the output video is.
    # So the args would be -vf setpts=0.02*PTS -r 30 in that case. 
    # x4=0.25, x8=0.125, x16=0.0625
    timelapse_args: -vf setpts=0.125*PTS

motion:
  threshold: 50 # 0-255 default=30 Less is more sensitive (was using 150 for months)
  improve_contrast: true

snapshots:
 enabled: True
 retain:
   default: 10 #days

detect:
  enabled: true
  fps: 5

objects:
  track:
    - person
    - cat
  filters:
    person:
      threshold: 0.8
      min_score: 0.65
    # cat:
    #   threshold: 0.8

review:
  alerts:
    labels:
     - bark

audio:
  enabled: true
  listen:
    - scream
    - yell
    - laughter

#birdseye:
#  enabled: True
#  mode: continuous

#ui:
  #live_mode: webrtc
camera_groups:
  Outdoors:
    order: 2
    icon: LuTrees
    cameras:
      - backgarden
      - doorbell
  Indoors:
    order: 1
    icon: LuHome
    cameras:
      - bedroom
      - livingroom
      - reolink
version: 0.14

Relevant log output

NA

FFprobe output from your camera

NA

Frigate stats

NA

Operating system

HassOS

Install method

HassOS Addon

Coral version

CPU (no coral)

Network connection

Wireless

Camera make and model

Foscam R5, Reolink E1 Pro, Reolink Doorbell WiFi

Any other information that may be helpful

No response

[hawkeye217]

If you're not trying to track extremely small or distant objects, the lowest resolution available for each of your substreams should do just fine for the detect role.

When possible, set your keyframe interval to be the same as your frame rate.

Bitrate shouldn't matter terribly, but if you see artifacts or "tears" in your stream, you could increase it until you don't.

[hawkeye217]

Have your camera output AAC audio on whatever stream (main or substream) you'd like, and then use that stream for audio detection. You don't need Opus set up in go2rtc for anything other than the cams you want to do two way talk with.

[Chaoscontrol]

Got it. I thought webRTC live view was slightly better than MSE, even if not using 2-way audio. Was that wrong?

That's why I had Opus for all, since 0.13 seemed to default to webRTC if it was able to. It's true that 0.14 doesn't tell you what it's using, so if it's using MSE using Opus is useless now. Is that what you mean?

[hawkeye217]

There's a very marginal difference between MSE and WebRTC, but the only time Frigate 0.14 will use WebRTC is if you're using two way talk.

MSE works out of the box for most setups and does not require extra setup, opening ports, etc.

So yep, no need to use Opus for cams that don't use two way talk.

[Chaoscontrol]

Amazing. I switched all 5 cameras from main stream to sub stream for audio and detection, CPU usage went down from 56-60% to 30% lol. And GPU from 30% to 13% or so for now. So happy right now. Never had numbers these low.

[hawkeye217]

Sounds about right. Good work!