[Detector Support]: OpenVINO seems broken in 0.14 dev builds

[leccelecce]

Describe the problem you are having

OpenVINO detector fails with logs below in 0.14. Seems to work fine in 0.13.2 in my production instance.

The OpenVINO config is copied directly from the docs.

I haven't posted full config for my dev instance but if someone else says that OpenVINO works in their 0.14 build, happy to do some more digging.

Version

0.14.0-28dd871d (May 2nd)

Frigate config file

...

detectors:

  openvino:
    type: openvino
    device: CPU
    model:
      path: /openvino-model/ssdlite_mobilenet_v2.xml

model:
  width: 300
  height: 300
  input_tensor: nhwc
  input_pixel_format: bgr
  labelmap_path: /openvino-model/coco_91cl_bkgr.txt

...

docker-compose file or Docker CLI command

services:

  frigate:
    image: frigate:latest
    container_name: frigate-dev
    pull_policy: never # local build, don't try and pull
    privileged: true # this may not be necessary for all setups
    restart: unless-stopped
    shm_size: "512mb"
    deploy:
      resources:
        limits:
          memory: 4096M
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /appdata/data/frigate-dev/config/:/config #DEV
      - /mnt/nas/nfs-nvr-share/frigate-dev/:/media/frigate #DEV
      - type: tmpfs # reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 536870912
    ports:
      - "5001:5000"
      - "18555:8555"
#      - "8555:8555/tcp"
#      - "8555:8555/udp"
#    devices:
#      - /dev/bus/usb/002:/dev/bus/usb/002
#      - /dev/dri/renderD128:/dev/dri/renderD128

Relevant log output

2024-04-30 14:44:48.608628489  [2024-04-30 14:44:48] frigate.detectors.plugins.openvino INFO    : Model Input Shape: [1,300,300,3]
2024-04-30 14:44:48.608808854  [2024-04-30 14:44:48] frigate.detectors.plugins.openvino INFO    : Model Output-0 Shape: [1,1,100,7]
2024-04-30 14:44:48.608942348  [2024-04-30 14:44:48] frigate.detectors.plugins.openvino INFO    : Model has 1 Output Tensors
2024-04-30 14:44:51.742996138  Process detector:openvino:
2024-04-30 14:44:51.743948942  Traceback (most recent call last):
2024-04-30 14:44:51.743976127    File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
2024-04-30 14:44:51.743978993      self.run()
2024-04-30 14:44:51.743985243    File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
2024-04-30 14:44:51.743989082      self._target(*self._args, **self._kwargs)
2024-04-30 14:44:51.744039295    File "/opt/frigate/frigate/object_detection.py", line 125, in run_detector
2024-04-30 14:44:51.744042012      detections = object_detector.detect_raw(input_frame)
2024-04-30 14:44:51.744044747    File "/opt/frigate/frigate/object_detection.py", line 75, in detect_raw
2024-04-30 14:44:51.744047342      return self.detect_api.detect_raw(tensor_input=tensor_input)
2024-04-30 14:44:51.744050356    File "/opt/frigate/frigate/detectors/plugins/openvino.py", line 84, in detect_raw
2024-04-30 14:44:51.744077466      infer_request.infer([tensor_input])
2024-04-30 14:44:51.744080596    File "/usr/local/lib/python3.9/dist-packages/openvino/runtime/ie_api.py", line 152, in infer
2024-04-30 14:44:51.744084024      return super().infer(normalize_inputs(self, dict(enumerate(inputs))))
2024-04-30 14:44:51.744086715  RuntimeError: The input blob size is not equal to the network input size: got 307200 expecting 270000

...followed later by...

2024-04-30 14:48:16.810154735  [2024-04-30 14:48:16] frigate.object_processing      INFO    : Exiting object processor...
2024-04-30 14:48:17.271767713  Exception in thread detection_proxy:
2024-04-30 14:48:17.271779499  Traceback (most recent call last):
2024-04-30 14:48:17.271782932    File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
2024-04-30 14:48:17.271785564      self.run()
2024-04-30 14:48:17.271788672    File "/opt/frigate/frigate/comms/detections_updater.py", line 37, in run
2024-04-30 14:48:17.271791948      zmq.proxy_steerable(
2024-04-30 14:48:17.271795227    File "_zmq.py", line 1685, in zmq.backend.cython._zmq.proxy_steerable
2024-04-30 14:48:17.271798018    File "_zmq.py", line 160, in zmq.backend.cython._zmq._check_rc
2024-04-30 14:48:17.271801061  zmq.error.ZMQError: Operation not supported

Operating system

Debian

Install method

Docker Compose

Coral version

USB

Any other information that may be helpful

No response

[leccelecce]

The input blob size is not equal to the network input size: got 307200 expecting 270000

This suggests it thinks the model input is 320x320 not 300x300, but I'm not clear why this would happen.

I do get the following in the logs prior to the error, which is the same as in 0.13.2, but I'm not sure if that's read from the actual model file by OpenVINO or just from the detector config.

2024-05-02 10:34:15.729911314  [2024-05-02 10:34:15] frigate.detectors.plugins.openvino INFO    : Model Input Shape: [1,300,300,3]
2024-05-02 10:34:15.730111115  [2024-05-02 10:34:15] frigate.detectors.plugins.openvino INFO    : Model Output-0 Shape: [1,1,100,7]
2024-05-02 10:34:15.730248418  [2024-05-02 10:34:15] frigate.detectors.plugins.openvino INFO    : Model has 1 Output Tensors

[NickM-27]

Add

      path: /openvino-model/ssdlite_mobilenet_v2.xml

To the main model config and remove the model config from the detector

[banthungprong]

Thanks, working now!

detectors:
  openvino:
    type: openvino
    device: CPU

model:
  width: 300
  height: 300
  input_tensor: nhwc
  input_pixel_format: bgr
  labelmap_path: /openvino-model/coco_91cl_bkgr.txt
  path: /openvino-model/ssdlite_mobilenet_v2.xml`

[NickM-27]

thanks for confirming, need to see why this is not working correctly