What is reasonable CPU usage for detection with OpenVINO and VAAPI?

[bobloadmire]

I have 5 1080p streams, using an Intel 8505 (1 performance core, 4 efficiency cores) with OpenVINO (5fps) and VAAPI. i'm sitting around 50% cpu usage, which seems high considered all of the hardware acceleration. Is there any tuning I can do to reduce idle usage?f

[NickM-27]

it depends on a lot of factors, if you are running detect at 1080, with go2rtc restreaming, etc. then that may not be unreasonable for a lower power CPU such as that. Best way to understand what is actually having an impact on CPU usage is a screenshot of the system page in the UI

[bobloadmire]

https://i.imgur.com/fyzH0La.png

screenshot of system page.

config:

mqtt:
  enabled: false
cameras:
  Driveway:
    ffmpeg:
      inputs:
        - path: rtsp:https://xx:[email protected]:554/stream1
          roles:
            - detect
            - clips
            - record
    birdseye:
      mode: motion
  Lyras_Room:
    ffmpeg:
      inputs:
        - path: rtsp:https://xx:[email protected]:554/stream1
          roles:
            - detect
            - clips
            - record
    birdseye:
      mode: motion
  Kitchen:
    ffmpeg:
      inputs:
        - path: rtsp:https://xxx:[email protected]:554/stream1
          roles:
            - detect
            - clips
            - record
    birdseye:
      mode: motion
  Hot_Tub:
    ffmpeg:
      inputs:
        - path: rtsp:https://xxxx:[email protected]:554/stream1
          roles:
            - detect
            - clips
            - record
    birdseye:
      mode: motion
  Side_Yard:
    ffmpeg:
      inputs:
        - path: rtsp:https://xxxx:[email protected]:554/stream1
          roles:
            - detect
            - clips
            - record
    birdseye:
      mode: motion         
ffmpeg:
  hwaccel_args: preset-vaapi
detectors:
  ov:
    type: openvino
    device: AUTO
    model:
      path: /openvino-model/FP16/ssdlite_mobilenet_v2.xml
model:
  width: 300
  height: 300
  input_tensor: nhwc
  input_pixel_format: bgr
  labelmap_path: /openvino-model/coco_91cl_bkgr.txt
record:
  enabled: True
  retain:
    days: 3
    mode: motion
  events:
    retain:
      default: 30
      mode: motion
objects:
  # Optional: list of objects to track from labelmap.txt (default: shown below)
  track:
    - person
    - car
    - dog
  # Optional: mask to prevent all object types from being detected in certain areas (default: no mask)
  # Checks based on the bottom center of the bounding box of the object.
  # NOTE: This mask is COMBINED with the object type specific mask below
  #mask: 0,0,1000,0,1000,200,0,200
  # Optional: filters to reduce false positives for specific object types
  filters:
    person:
      # Optional: minimum width*height of the bounding box for the detected object (default: 0)
      #min_area: 5000
      # Optional: maximum width*height of the bounding box for the detected object (default: 24000000)
      #max_area: 100000
      # Optional: minimum width/height of the bounding box for the detected object (default: 0)
      #min_ratio: 0.5
      # Optional: maximum width/height of the bounding box for the detected object (default: 24000000)
      #max_ratio: 2.0
      # Optional: minimum score for the object to initiate tracking (default: shown below)
      min_score: 0.5
      # Optional: minimum decimal percentage for tracked object's computed score to be considered a true positive (default: shown below)
      threshold: 0.75
      # Optional: mask to prevent this object type from being detected in certain areas (default: no mask)
      # Checks based on the bottom center of the bounding box of the object
      #mask: 0,0,1000,0,1000,200,0,200

# Optional: Motion configuration
# NOTE: Can be overridden at the camera level
detect:
  # Optional: width of the frame for the input with the detect role (default: use native stream resolution)
  width: 1920
  # Optional: height of the frame for the input with the detect role (default: use native stream resolution)
  height: 1080
  # Optional: desired fps for your camera for the input with the detect role (default: shown below)
  # NOTE: Recommended value of 5. Ideally, try and reduce your FPS on the camera.
  fps: 5
  # Optional: enables detection for the camera (default: True)
  enabled: True

# Optional: Configuration for the jpg snapshots written to the clips directory for each event
# NOTE: Can be overridden at the camera level
# Include all cameras by default in Birdseye view
birdseye:
  enabled: false
  mode: continuous

[bobloadmire]

OK, good to know. I assumed that if I was using CPU detection my inference latency would be much higher, but is that not the case? 11-12ms seems decent? Is there a defined fix for the intel issue, or is it up in the air?

[NickM-27]

openvino has optimizations for intel platform that apply to both CPU and GPU, though GPU will be faster. The fix is to apply the two environment variables in the post I linked

[bobloadmire]

this is my frigate config, do you have any advice as to where I add the env var?

arch: amd64
features: nesting=1
hostname: frigate
memory: 2048
mp0: /mnt/lxc_shares/coopnet/frigate/,mp=/media/frigate
net0: name=eth0,bridge=vmbr0,gw=192.168.0.1,hwaddr=BC:24:11:74:AF:06,ip=192.168.0.14/24,type=veth
onboot: 1
ostype: debian
rootfs: local:102/vm-102-disk-1.raw,size=20G
startup: order=20
swap: 2048
tags: proxmox-helper-scripts
lxc.cgroup2.devices.allow: a
lxc.cap.drop:
lxc.cgroup2.devices.allow: c 188:* rwm
lxc.cgroup2.devices.allow: c 189:* rwm
lxc.mount.entry: /dev/serial/by-id dev/serial/by-id none bind,optional,create=dir
lxc.mount.entry: /dev/ttyUSB0 dev/ttyUSB0 none bind,optional,create=file
lxc.mount.entry: /dev/ttyUSB1 dev/ttyUSB1 none bind,optional,create=file
lxc.mount.entry: /dev/ttyACM0 dev/ttyACM0 none bind,optional,create=file
lxc.mount.entry: /dev/ttyACM1 dev/ttyACM1 none bind,optional,create=file
lxc.cgroup2.devices.allow: c 226:0 rwm
lxc.cgroup2.devices.allow: c 226:128 rwm
lxc.cgroup2.devices.allow: c 29:0 rwm
lxc.mount.entry: /dev/fb0 dev/fb0 none bind,optional,create=file
lxc.mount.entry: /dev/dri dev/dri none bind,optional,create=dir
lxc.mount.entry: /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,create=file

[NickM-27]

No, I have no experience with proxmox

[bobloadmire]

ok thanks, i'll figure it out

[bobloadmire]

Also confirming, setting detect enabled: false drops cpu usage to 5-10%

[bobloadmire]

Ok im going to add this info here for other proxmox users:

How to add envirmonent variables to an LXC.

open the LXC.conf file (usually located at /etc/pve/lxc/<container_id>.conf) and add your variables like this:
lxc.environment = VARIABLE_NAME=value

so I added mine,

lxc.environment: NEOReadDebugKeys=1
lxc.environment: OverrideGpuAddressSpace=48

and it actually made my situation worse, my CPU usage climbed up to 70% idle. so not sure what happened, but this solution didn't work for me.