[Detector Support]: Openvino - Is it really so good or there is something i don't understand

[koukobin79]

Describe the problem you are having

My device is the HP Elitedesk 800 G3 Mini (65W version), with i5-6500 cpu, 16GB Ram and 256GB SSD. I am running Home Assistant in a VM on top of the Proxmon (ver 8) hypervisor (along another one VM). The Frigate is running as add-on of Home Assistant. I have the iGPU (HD 530) passthough to home assistant by using this guide: https://3os.org/infrastructure/proxmox/gpu-passthrough/igpu-passthrough-to-vm/ , so i am using hardware acceleration and the openvino detector. Everything seems to be working fine.

My config until now has only one camera and as i see the inference speed is around 15ms.
Correct if i am wrong, but this means that this system can hold up to 13 cameras if the detection is running at 5 frames per second. Can this be true without using the Coral? Is there something i don't understand?

Version

0.13.2-6476F8A

Frigate config file

mqtt:
  host: xxx.xxx.xxx.xxx
  user: *******
  password: ******

ffmpeg:
  hwaccel_args: preset-vaapi

detectors:
  ov:
    type: openvino
    device: AUTO
    model:
      path: /openvino-model/ssdlite_mobilenet_v2.xml

model:
  width: 300
  height: 300
  input_tensor: nhwc
  input_pixel_format: bgr
  labelmap_path: /openvino-model/coco_91cl_bkgr.txt

birdseye:
  enabled: True
  mode: continuous

go2rtc:
  streams:
    garden-HD: 
      - ffmpeg:rtsp:https://user:password@ipofcamer:554/stream1
      - "ffmpeg:Garden#audio=aac"
    garden:
      - ffmpeg:rtsp:https://user:password!@ipofcamera:554/stream2 

cameras:
  garden:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs: 
       - path:  rtsp:https://127.0.0.1:8554/garden-HD
         input_args: preset-rtsp-restream
         roles:
            - record
       - path: rtsp:https://127.0.0.1:8554/garden
         input_args: preset-rtsp-restream
         roles:
            - detect

    detect:
      width: 640
      height: 360
      fps: 5

    record:
      enabled: True
      events:
          retain:
            default: 5    

    snapshots:
      enabled: true
      timestamp: false

docker-compose file or Docker CLI command

I am not using docker.

Relevant log output

2024-03-09 18:04:35.083533927  [INFO] Preparing Frigate...
2024-03-09 18:04:35.097846699  [INFO] Starting Frigate...
2024-03-09 18:04:36.391988643  [2024-03-09 21:04:36] frigate.app                    INFO    : Starting Frigate (0.13.2-6476f8a)
2024-03-09 18:04:36.392254032  [2024-03-09 21:04:36] frigate.app                    INFO    : Creating directory: /tmp/cache
2024-03-09 18:04:36.411732651  [2024-03-09 21:04:36] frigate.config                 WARNING : Customizing more than a detector model path is unsupported.
2024-03-09 18:04:36.419169196  [2024-03-09 21:04:36] peewee_migrate.logs            INFO    : Starting migrations
2024-03-09 18:04:36.428083535  [2024-03-09 21:04:36] peewee_migrate.logs            INFO    : There is nothing to migrate
2024-03-09 18:04:36.428087297  [2024-03-09 21:04:36] frigate.app                    INFO    : Recording process started: 307
2024-03-09 18:04:36.430865013  [2024-03-09 21:04:36] frigate.app                    INFO    : go2rtc process pid: 89
2024-03-09 18:04:36.457823704  [2024-03-09 21:04:36] frigate.app                    INFO    : Output process started: 319
2024-03-09 18:04:36.468697583  [2024-03-09 21:04:36] detector.ov                    INFO    : Starting detection process: 317
2024-03-09 18:04:36.485948808  [2024-03-09 21:04:36] frigate.app                    INFO    : Camera processor started for garden: 325
2024-03-09 18:04:36.489776671  [2024-03-09 21:04:36] frigate.app                    INFO    : Capture process started for garden: 331
2024-03-09 18:04:36.750531733  [2024-03-09 21:04:36] frigate.detectors.plugins.openvino INFO    : Model Input Shape: [1,300,300,3]
2024-03-09 18:04:36.750613220  [2024-03-09 21:04:36] frigate.detectors.plugins.openvino INFO    : Model Output-0 Shape: [1,1,100,7]
2024-03-09 18:04:36.750662177  [2024-03-09 21:04:36] frigate.detectors.plugins.openvino INFO    : Model has 1 Output Tensors

Operating system

HassOS

Install method

HassOS Addon

Coral version

Other

Any other information that may be helpful

No response

[NickM-27]

Correct if i am wrong, but this means that this system can hold up to 13 cameras if the detection is running at 5 frames per second.

that is mostly right. Technically it is not that many because frigate will often run detection multiple times on the same frame.

But yes, this is correct, it is performing with similar performance (just a tad slower) as the coral

[koukobin79]

Thanks! I will not be using 13 cameras of course. The plan is around six so i will be ok. I couldn't imagine that without Coral i would be able to do that. Thanks again.

[NickM-27]

The advantage of the coral is performance for power draw, the GPU will definitely use more power but that doesn't matter to everyone.

Frigate supports many different types of detectors.

[DReffects]

I was impressed by how well this works. I am running HA on a n100 mini PC with the P2 limit Set to 18W.
Without frigate, HA was using 7.5W average.
With frigate, openvino and 4 cameras at 720p 5fps for object detection plus uhd recording the system only uses 4W more. FOUR.