[Support]: Addition of Verisilicon TPU to Frigate - working PoC ; how to incorporate - [Diff included] #5890

RichardPar · 2023-04-01T16:59:19Z

Describe the problem you are having

https://github.com/blakeblackshear/frigate/compare/v0.11.1...RichardPar:frigate:v0.11.1_DEV_VIM3?expand=1

How to add a new TPU to frigate in a good manner - I have changed the v0.11.1 code to replace the Coral TPU with a Verisilicon one. Its currently working (docker needs --priveledged mode)

What would be a good way to add to Frigate? to add Verisilicon on VIM3 board)

Current Inference time is about 23mSeconds

Version

v0.11.1

Frigate config file

None

Relevant log output

khadas@Khadas:~/git/frigate$ docker run --privileged -v "$(pwd)"/config:/config -v "$(pwd)"/config:/media frigate
[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[2023-04-01 17:07:12] frigate.app                    INFO    : Starting Frigate (0.11.1-ad7be18)
[2023-04-01 17:07:12] frigate.app                    INFO    : Creating directory: /tmp/cache
Starting migrations
[2023-04-01 17:07:12] peewee_migrate                 INFO    : Starting migrations
There is nothing to migrate
[2023-04-01 17:07:12] peewee_migrate                 INFO    : There is nothing to migrate
[2023-04-01 17:07:12] detector.coral                 INFO    : Starting detection process: 221
[2023-04-01 17:07:12] frigate.app                    INFO    : Output process started: 222
[2023-04-01 17:07:12] frigate.edgetpu                INFO    : Attempting to load TPU as usb
[2023-04-01 17:07:12] ws4py                          INFO    : Using epoll
Vx delegate: allowed_cache_mode set to 0.
Vx delegate: device num set to 0.
Vx delegate: allowed_builtin_code set to 0.
Vx delegate: error_during_init set to 0.
Vx delegate: error_during_prepare set to 0.
Vx delegate: error_during_invoke set to 0.
[2023-04-01 17:07:12] frigate.app                    INFO    : Camera processor started for test: 227
WARNING: Fallback unsupported op 32 to TfLite
[2023-04-01 17:07:12] frigate.app                    INFO    : Capture process started for test: 229
[2023-04-01 17:07:13] frigate.edgetpu                INFO    : TPU found
[2023-04-01 17:07:13] ws4py                          INFO    : Using epoll

FFprobe output from your camera

None

Frigate stats

{"test": {"camera_fps": 5.1, "process_fps": 5.1, "skipped_fps": 0.0, "detection_fps": 0.0, "pid": 226, "capture_pid": 228}, "detectors": {"coral": {"inference_speed": 26.87, "detection_start": 0.0, "pid": 220}}, "detection_fps": 0.0, "service": {"uptime": 26115, "version": "0.11.1-ad7be18", "latest_version": "0.11.1", "storage": {"/media/frigate/recordings": {"total": 30970.2, "used": 13870.7, "free": 16748.0, "mount_type": "ext4"}, "/media/frigate/clips": {"total": 30970.2, "used": 13870.7, "free": 16748.0, "mount_type": "ext4"}, "/tmp/cache": {"total": 30970.2, "used": 13870.7, "free": 16748.0, "mount_type": "overlay"}, "/dev/shm": {"total": 67.1, "used": 3.7, "free": 63.4, "mount_type": "tmpfs"}}, "temperatures": {}}}

QoS: 0 - Retain: false

Operating system

Debian

Install method

Docker Compose

Coral version

USB

Network connection

Wired

Camera make and model

None

Any other information that may be helpful

No response

The text was updated successfully, but these errors were encountered:

NickM-27 · 2023-04-01T17:09:30Z

If suggest looking at the 0.12 code (dev branch) which has many examples of how to setup the detectors

RichardPar · 2023-04-01T17:36:42Z

If suggest looking at the 0.12 code (dev branch) which has many examples of how to setup the detectors

Thanks, is there a small write-up on how to select different detectors on 0.12? OpenVino etc etc.. just trying to understand how the config.yml tags get to the python code and select the appropriate NPU

For Coral

detectors:
  coral:
    type: edgetpu
    device: usb

NickM-27 · 2023-04-01T17:41:08Z

There is no write up, just the code. You can see that logic here: https://github.com/blakeblackshear/frigate/blob/dev/frigate/detectors/detector_types.py where each of the detector types gets added if it is found in the config

You can see an implementation of a detector here: https://github.com/blakeblackshear/frigate/blob/dev/frigate/detectors/plugins/edgetpu_tfl.py

RichardPar · 2023-04-01T17:45:57Z

There is no write up, just the code. You can see that logic here: https://github.com/blakeblackshear/frigate/blob/dev/frigate/detectors/detector_types.py where each of the detector types gets added if it is found in the config

You can see an implementation of a detector here: https://github.com/blakeblackshear/frigate/blob/dev/frigate/detectors/plugins/edgetpu_tfl.py

Thanks again.. excuse my ignorance :D (python is not my thing usually) .. the plan is to copy the Coral one and make a new plugin... was just trying to see how thigs tied together

NickM-27 · 2023-04-01T17:57:56Z

I'd suggest just doing a draft PR, it'll be easier to put pointers there.

RichardPar · 2023-04-01T18:05:51Z

I grabbed the DEV branch... and it seems to fail running the ov-converter on my deb11 ARM

khadas@Khadas:~/git/frigate$ make
echo 'VERSION = "0.12.0-1b8cd10"' > frigate/version.py
docker buildx build --target=frigate --tag frigate:latest --load .
[+] Building 3.7s (22/61)
 => [internal] load build definition from Dockerfile                                                                                                                           0.0s
 => => transferring dockerfile: 32B                                                                                                                                            0.0s
 => [internal] load .dockerignore                                                                                                                                              0.0s
 => => transferring context: 34B                                                                                                                                               0.0s
 => resolve image config for docker.io/docker/dockerfile:1.2                                                                                                                   0.4s
 => CACHED docker-image:https://docker.io/docker/dockerfile:1.2@sha256:e2a8561e419ab1ba6b2fe6cbdf49fd92b95912df1cf7d313c3e2230a333fdbcc                                              0.0s
 => [internal] load metadata for docker.io/library/debian:11                                                                                                                   0.4s
 => [internal] load metadata for docker.io/library/debian:11                                                                                                                   0.4s
 => [internal] load metadata for docker.io/library/debian:11-slim                                                                                                              0.4s
 => [internal] load metadata for docker.io/library/node:16                                                                                                                     0.4s
 => [rootfs 1/4] WORKDIR /opt/frigate/                                                                                                                                         0.0s
 => CACHED [slim-base 1/1] FROM docker.io/library/debian:11-slim@sha256:7acda01e55b086181a6fa596941503648e423091ca563258e2c1657d140355b1                                       0.0s
 => [internal] load build context                                                                                                                                              0.1s
 => => transferring context: 19.86kB                                                                                                                                           0.1s
 => CANCELED [web-build 1/6] FROM docker.io/library/node:16@sha256:051b68ed341fee26f4c08b15815ac4bff6a5351c68e7b33c66912677cc4be7ac                                            1.9s
 => => resolve docker.io/library/node:16@sha256:051b68ed341fee26f4c08b15815ac4bff6a5351c68e7b33c66912677cc4be7ac                                                               0.0s
 => => sha256:051b68ed341fee26f4c08b15815ac4bff6a5351c68e7b33c66912677cc4be7ac 776B / 776B                                                                                     0.0s
 => => sha256:2a6a4c07cfbed3ee02ce3ca986af5b896058d0632d0101f64fecb33ccdd11561 2.21kB / 2.21kB                                                                                 0.0s
 => => sha256:3e37868ebf669334a2dbdb206ac7b84d8f8d184a40dfb8c9bc501b29cda12548 8.39MB / 10.00MB                                                                                1.8s
 => => sha256:26ed31aaee8c72a23f99458066bef1b173830b587d33b73e080c444c3a56fa0c 7.56kB / 7.56kB                                                                                 0.0s
 => => sha256:4e2befb7f5d18aa27b3619ddf1b93607e62ca82d0c627557537c149893346d86 5.24MB / 50.45MB                                                                                1.8s
 => => sha256:792af667f62688dbef1f3ffeeca2daa6d448a62b6cacd604f1c4fc043d6cc2a6 0B / 7.86MB                                                                                     1.8s
 => CACHED [base 1/1] FROM docker.io/library/debian:11@sha256:7b991788987ad860810df60927e1adbaf8e156520177bd4db82409f81dd3b721                                                 0.0s
 => [base_amd64 1/1] FROM docker.io/library/debian:11@sha256:7b991788987ad860810df60927e1adbaf8e156520177bd4db82409f81dd3b721                                                  0.0s
 => CANCELED [wheels 1/7] RUN apt-get -qq update     && apt-get -qq install -y     apt-transport-https     gnupg     wget     && apt-key adv --keyserver keyserver.ubuntu.com  1.7s
 => CANCELED [wget 1/2] RUN apt-get update     && apt-get install -y wget xz-utils     && rm -rf /var/lib/apt/lists/*                                                          1.5s
 => CACHED [rootfs 2/4] COPY frigate frigate/                                                                                                                                  0.0s
 => CACHED [rootfs 3/4] COPY migrations migrations/                                                                                                                            0.0s
 => CANCELED [deps 1/4] RUN --mount=type=bind,source=docker/install_deps.sh,target=/deps/install_deps.sh     /deps/install_deps.sh                                             1.6s
 => CACHED [ov-converter 1/3] COPY requirements-ov.txt /requirements-ov.txt                                                                                                    0.0s
 => ERROR [ov-converter 2/3] RUN apt-get -qq update     && apt-get -qq install -y wget python3 python3-distutils     && wget -q https://bootstrap.pypa.io/get-pip.py -O get-p  1.2s
 => CANCELED [nginx 1/1] RUN --mount=type=tmpfs,target=/tmp --mount=type=tmpfs,target=/var/cache/apt     --mount=type=bind,source=docker/build_nginx.sh,target=/deps/build_ng  1.4s
------
 > [ov-converter 2/3] RUN apt-get -qq update     && apt-get -qq install -y wget python3 python3-distutils     && wget -q https://bootstrap.pypa.io/get-pip.py -O get-pip.py     && python3 get-pip.py "pip"     && pip install -r /requirements-ov.txt:
#0 1.087 standard_init_linux.go:219: exec user process caused: exec format error
------
ERROR: failed to solve: executor failed running [/bin/sh -c apt-get -qq update     && apt-get -qq install -y wget python3 python3-distutils     && wget -q https://bootstrap.pypa.io/get-pip.py -O get-pip.py     && python3 get-pip.py "pip"     && pip install -r /requirements-ov.txt]: exit code: 1
make: *** [Makefile:13: local] Error 1
khadas@Khadas:~/git/frigate$

NickM-27 · 2023-04-01T18:15:37Z

Yeah, it needs to be built in qemu since it's only amd64. If you're not working on the ov detector I'd suggest just commenting out the ov-converter steps

RichardPar · 2023-04-01T18:21:12Z

Next error :(

Got past it by commenting out the platform amd64 bit in the Dockerfile

khadas@Khadas:~/git/frigate$ make
echo 'VERSION = "0.12.0-1b8cd10"' > frigate/version.py
docker buildx build --target=frigate --tag frigate:latest --load .
[+] Building 3.4s (21/54)
 => [internal] load build definition from Dockerfile                                                                                                                           0.0s
 => => transferring dockerfile: 32B                                                                                                                                            0.0s
 => [internal] load .dockerignore                                                                                                                                              0.0s
 => => transferring context: 34B                                                                                                                                               0.0s
 => resolve image config for docker.io/docker/dockerfile:1.2                                                                                                                   0.4s
 => CACHED docker-image:https://docker.io/docker/dockerfile:1.2@sha256:e2a8561e419ab1ba6b2fe6cbdf49fd92b95912df1cf7d313c3e2230a333fdbcc                                              0.0s
 => [internal] load metadata for docker.io/library/node:16                                                                                                                     0.4s
 => [internal] load metadata for docker.io/library/debian:11-slim                                                                                                              0.4s
 => [internal] load metadata for docker.io/library/debian:11                                                                                                                   0.4s
 => [rootfs 1/4] WORKDIR /opt/frigate/                                                                                                                                         0.0s
 => [internal] load build context                                                                                                                                              0.1s
 => => transferring context: 20.57kB                                                                                                                                           0.1s
 => CACHED [slim-base 1/1] FROM docker.io/library/debian:11-slim@sha256:7acda01e55b086181a6fa596941503648e423091ca563258e2c1657d140355b1                                       0.0s
 => [web-build 1/6] FROM docker.io/library/node:16@sha256:051b68ed341fee26f4c08b15815ac4bff6a5351c68e7b33c66912677cc4be7ac                                                     0.0s
 => CACHED [base 1/1] FROM docker.io/library/debian:11@sha256:7b991788987ad860810df60927e1adbaf8e156520177bd4db82409f81dd3b721                                                 0.0s
 => CANCELED [wheels 1/7] RUN apt-get -qq update     && apt-get -qq install -y     apt-transport-https     gnupg     wget     && apt-key adv --keyserver keyserver.ubuntu.com  1.4s
 => CANCELED [wget 1/2] RUN apt-get update     && apt-get install -y wget xz-utils     && rm -rf /var/lib/apt/lists/*                                                          1.3s
 => CACHED [rootfs 2/4] COPY frigate frigate/                                                                                                                                  0.0s
 => CACHED [rootfs 3/4] COPY migrations migrations/                                                                                                                            0.0s
 => CANCELED [deps 1/4] RUN --mount=type=bind,source=docker/install_deps.sh,target=/deps/install_deps.sh     /deps/install_deps.sh                                             1.3s
 => CANCELED [nginx 1/1] RUN --mount=type=tmpfs,target=/tmp --mount=type=tmpfs,target=/var/cache/apt     --mount=type=bind,source=docker/build_nginx.sh,target=/deps/build_ng  1.3s
 => CACHED [web-build 2/6] WORKDIR /work                                                                                                                                       0.0s
 => CACHED [web-build 3/6] COPY web/package.json web/package-lock.json ./                                                                                                      0.0s
 => ERROR [web-build 4/6] RUN npm install                                                                                                                                      1.1s
------
 > [web-build 4/6] RUN npm install:
#0 0.970 standard_init_linux.go:219: exec user process caused: exec format error
------
ERROR: failed to solve: executor failed running [/bin/sh -c npm install]: exit code: 1
make: *** [Makefile:13: local] Error 1
khadas@Khadas:~/git/frigate$

RichardPar · 2023-04-01T18:56:16Z

Done.. will raise a PR

{"before": {"id": "1680375244.063041-ia7vkl", "camera": "test", "frame_time": 1680375250.884574, "snapshot_time": 1680375246.463054, "label": "person", "sub_label": null, "top_score": 0.83984375, "false_positive": false, "start_time": 1680375244.063041, "end_time": null, "score": 0.70703125, "box": [756, 882, 898, 1078], "area": 27832, "ratio": 0.7244897959183674, "region": [530, 500, 1110, 1080], "stationary": false, "motionless_count": 2, "position_changes": 1, "current_zones": [], "entered_zones": [], "has_clip": false, "has_snapshot": false}, "after": {"id": "1680375244.063041-ia7vkl", "camera": "test", "frame_time": 1680375251.251274, "snapshot_time": 1680375246.463054, "label": "person", "sub_label": null, "top_score": 0.83984375, "false_positive": false, "start_time": 1680375244.063041, "end_time": 1680375256.474941, "score": 0.5859375, "box": [619, 961, 969, 1079], "area": 41300, "ratio": 2.9661016949152543, "region": [557, 576, 1061, 1080], "stationary": false, "motionless_count": 0, "position_changes": 1, "current_zones": [], "entered_zones": [], "has_clip": false, "has_snapshot": false}, "type": "end"}

github-actions · 2023-05-02T00:09:05Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

RichardPar added support triage labels Apr 1, 2023

NickM-27 added the beta Related to the current beta version of frigate label Apr 1, 2023

NickM-27 linked a pull request Apr 5, 2023 that will close this issue

Adding VIM3 (Verisilicon TPU) to Frigate #5892

Closed

github-actions bot added the stale label May 2, 2023

github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale May 6, 2023

Awei2020 mentioned this issue Oct 28, 2023

[Snyk] Security upgrade axios from 1.3.5 to 1.6.0 Awei2020/frigate#7

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Support]: Addition of Verisilicon TPU to Frigate - working PoC ; how to incorporate - [Diff included] #5890

[Support]: Addition of Verisilicon TPU to Frigate - working PoC ; how to incorporate - [Diff included] #5890

RichardPar commented Apr 1, 2023 •

edited

Loading

NickM-27 commented Apr 1, 2023

RichardPar commented Apr 1, 2023

NickM-27 commented Apr 1, 2023

RichardPar commented Apr 1, 2023

NickM-27 commented Apr 1, 2023

RichardPar commented Apr 1, 2023

NickM-27 commented Apr 1, 2023 •

edited

Loading

RichardPar commented Apr 1, 2023 •

edited

Loading

RichardPar commented Apr 1, 2023

github-actions bot commented May 2, 2023

[Support]: Addition of Verisilicon TPU to Frigate - working PoC ; how to incorporate - [Diff included] #5890

[Support]: Addition of Verisilicon TPU to Frigate - working PoC ; how to incorporate - [Diff included] #5890

Comments

RichardPar commented Apr 1, 2023 • edited Loading

Describe the problem you are having

Version

Frigate config file

Relevant log output

FFprobe output from your camera

Frigate stats

Operating system

Install method

Coral version

Network connection

Camera make and model

Any other information that may be helpful

NickM-27 commented Apr 1, 2023

RichardPar commented Apr 1, 2023

NickM-27 commented Apr 1, 2023

RichardPar commented Apr 1, 2023

NickM-27 commented Apr 1, 2023

RichardPar commented Apr 1, 2023

NickM-27 commented Apr 1, 2023 • edited Loading

RichardPar commented Apr 1, 2023 • edited Loading

RichardPar commented Apr 1, 2023

github-actions bot commented May 2, 2023

RichardPar commented Apr 1, 2023 •

edited

Loading

NickM-27 commented Apr 1, 2023 •

edited

Loading

RichardPar commented Apr 1, 2023 •

edited

Loading