PTZ is slow

[littlebritaly]

Describe the problem you are having

I installed Frigate yesterday on a new mini pc (Intel chip with 16gb ram).

After a lot of trial and error and reading of docs, i got PTZ working on a Topcony cctv camera (available on Amazon).

The system is to replace a mac mini running Security Spy (which is super reliable, no issues in five years), which would need replacing to add more cameras (i need to go from four to six cameras).

PTZ on the mac works flawlessly, but the PTZ on Frigate is incredibly slow, if it happens at all (same network, same camera)

I've ordered a "Google Coral USB Edge TPU ML Accelerator" (is this the right one ?) to take load off the cpu, but is there anything else I can do to speed up PTZ ?

Took me a while to figure out the ffmpeg command (and not sure it it's entiely correct), for brevity here is the command i used

ffprobe -rtsp_transport tcp -i rtsp:https://192.168.0.110:554/11 -select_streams v:0 -show_frames -show_entries frame=coded_picture_number,pkt_pts_time -of csv=p=0

Version

0.13.1-34fb1c2

Frigate config file

mqtt:
  enabled: False

record:
  enabled: True
  retain:
    days: 10
  events:
    retain:
      default: 10

ffmpeg:
  hwaccel_args: preset-vaapi

birdseye:
  enabled: True
  width: 1080
  height: 1920
  quality: 1
  mode: continuous





  


cameras:
  FrontDoor:
    enabled: True
    ffmpeg:
      inputs:
        - path: rtsp:https://192.168.0.215:554/11/
          roles:
            - detect
            - record
    detect:
      enabled: True
  Front2:
    enabled: True
    ffmpeg:
      inputs:
        - path: rtsp:https://192.168.0.2/user=admin_password=XXXXX_channel=1_stream=0./
          roles:
            - detect
            - record
    detect:
      enabled: True

  BackDoor:
    enabled: True
    ffmpeg:
      inputs:
        - path: rtsp:https://192.168.0.110:554/11/?proto=Onvif
          roles:
            - detect
            - record
    detect:
      enabled: True
    onvif:
      host: 192.168.0.110
      port: 8080
      user: XXXXXXX
      password: XXXXXXX

Relevant log output

2024-02-18 12:44:05.434650092  [2024-02-18 12:44:05] frigate.detectors              WARNING : CPU detectors are not recommended and should only be used for testing or for trial purposes.
2024-02-18 12:44:05.453278992  [2024-02-18 12:44:05] frigate.app                    INFO    : Camera processor started for FrontDoor: 331
2024-02-18 12:44:05.468121598  [2024-02-18 12:44:05] frigate.app                    INFO    : Camera processor started for Front2: 332
2024-02-18 12:44:05.468125693  [2024-02-18 12:44:05] frigate.app                    INFO    : Camera processor started for BackDoor: 333
2024-02-18 12:44:05.469261572  [2024-02-18 12:44:05] frigate.app                    INFO    : Capture process started for FrontDoor: 335
2024-02-18 12:44:05.475333934  [2024-02-18 12:44:05] frigate.app                    INFO    : Capture process started for Front2: 339
2024-02-18 12:44:05.482166329  [2024-02-18 12:44:05] frigate.app                    INFO    : Capture process started for BackDoor: 343
2024-02-18 12:44:05.983101806  [2024-02-18 12:44:05] frigate.app                    WARNING : The current SHM size of 120.0MB is too small, recommend increasing it to at least 131.7MB.
2024-02-18 12:44:45.483922628  [h264 @ 0x7f68e85e6040] left block unavailable for requested intra4x4 mode -1
2024-02-18 12:44:45.483926496  [h264 @ 0x7f68e85e6040] error while decoding MB 0 62, bytestream 1256
2024-02-18 12:44:45.488033375  [h264 @ 0x7f68e85e6040] error while decoding MB 14 47, bytestream -6
2024-02-18 12:48:30.524599631  [h264 @ 0x7f68e85c84c0] cabac decode of qscale diff failed at 59 61
2024-02-18 12:48:30.524603428  [h264 @ 0x7f68e85c84c0] error while decoding MB 59 61, bytestream 2246
2024-02-18 12:48:30.537497137  [h264 @ 0x7f68e85c84c0] error while decoding MB 116 46, bytestream -8
2024-02-18 13:09:20.710474148  [h264 @ 0x7f68e87213c0] error while decoding MB 82 31, bytestream -5
2024-02-18 14:30:35.965016101  [2024-02-18 14:30:35] frigate.output                 WARNING : Unable to copy frame Front21708260252.179099 to birdseye.
2024-02-18 14:30:35.966946511  [2024-02-18 14:30:35] frigate.output                 WARNING : Unable to copy frame FrontDoor1708260253.036174 to birdseye.
2024-02-18 14:45:32.767076525  [2024-02-18 14:45:32] frigate.comms.dispatcher       INFO    : Setting ptz command to move_right for BackDoor
2024-02-18 14:45:32.974229447  [2024-02-18 14:45:32] frigate.comms.dispatcher       INFO    : Setting ptz command to stop for BackDoor
2024-02-18 14:45:33.452512311  [2024-02-18 14:45:33] frigate.comms.dispatcher       INFO    : Setting ptz command to move_right for BackDoor
2024-02-18 14:45:34.264481417  [2024-02-18 14:45:34] frigate.comms.dispatcher       INFO    : Setting ptz command to stop for BackDoor
2024-02-18 14:45:34.636173019  [2024-02-18 14:45:34] frigate.comms.dispatcher       INFO    : Setting ptz command to move_right for BackDoor
2024-02-18 14:45:36.157741892  [2024-02-18 14:45:36] frigate.comms.dispatcher       INFO    : Setting ptz command to stop for BackDoor
2024-02-18 14:45:37.568239737  [2024-02-18 14:45:37] frigate.comms.dispatcher       INFO    : Setting ptz command to move_right for BackDoor
2024-02-18 14:45:37.767927245  [2024-02-18 14:45:37] frigate.comms.dispatcher       INFO    : Setting ptz command to stop for BackDoor
2024-02-18 14:45:37.925266090  [2024-02-18 14:45:37] frigate.comms.dispatcher       INFO    : Setting ptz command to move_down for BackDoor
2024-02-18 14:45:38.367468005  [2024-02-18 14:45:38] frigate.comms.dispatcher       INFO    : Setting ptz command to stop for BackDoor
2024-02-18 14:45:48.598831342  [2024-02-18 14:45:48] frigate.comms.dispatcher       INFO    : Setting ptz command to move_left for BackDoor
2024-02-18 14:45:50.020498935  [2024-02-18 14:45:50] frigate.comms.dispatcher       INFO    : Setting ptz command to stop for BackDoor
2024-02-18 14:45:51.483627796  [2024-02-18 14:45:51] frigate.comms.dispatcher       INFO    : Setting ptz command to move_left for BackDoor
2024-02-18 14:45:51.953198914  [2024-02-18 14:45:51] frigate.comms.dispatcher       INFO    : Setting ptz command to stop for BackDoor
2024-02-18 14:45:52.314015915  [2024-02-18 14:45:52] frigate.comms.dispatcher       INFO    : Setting ptz command to move_left for BackDoor
2024-02-18 14:45:52.530429936  [2024-02-18 14:45:52] frigate.comms.dispatcher       INFO    : Setting ptz command to stop for BackDoor
2024-02-18 14:45:55.841334328  [2024-02-18 14:45:55] frigate.comms.dispatcher       INFO    : Setting ptz command to preset for BackDoor
2024-02-18 15:23:58.301798875  [2024-02-18 15:23:58] frigate.video                  ERROR   : FrontDoor: Unable to read frames from ffmpeg process.
2024-02-18 15:23:58.302472084  [2024-02-18 15:23:58] frigate.video                  ERROR   : FrontDoor: ffmpeg process is not running. exiting capture thread...
2024-02-18 15:24:05.998312970  [2024-02-18 15:24:05] watchdog.FrontDoor             ERROR   : Ffmpeg process crashed unexpectedly for FrontDoor.
2024-02-18 15:24:06.008195428  [2024-02-18 15:24:05] watchdog.FrontDoor             ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2024-02-18 15:24:06.008845801  [2024-02-18 15:24:05] ffmpeg.FrontDoor.detect        ERROR   : [segment @ 0x55ce0e4c5b80] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
2024-02-18 15:24:06.009528412  [2024-02-18 15:24:05] ffmpeg.FrontDoor.detect        ERROR   : rtsp:https://192.168.0.215:554/11/: Connection timed out
2024-02-18 15:24:06.010225000  [2024-02-18 15:24:05] ffmpeg.FrontDoor.detect        ERROR   :     Last message repeated 1 times

FFprobe output from your camera

w_frames -show_entries frame=coded_picture_number,pkt_pts_time -of csv=p=0
ffprobe version 6.1.1 Copyright (c) 2007-2023 the FFmpeg developers
  built with gcc 13.2.1 (Alpine 13.2.1_git20231014) 20231014
  configuration: --prefix=/usr --disable-librtmp --disable-lzma --disable-static --disable-stripping --enable-avfilter --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-libmp3lame --enable-libopenmpt --enable-libopus --enable-libplacebo --enable-libpulse --enable-librav1e --enable-librist --enable-libsoxr --enable-libsrt --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-lto=auto --enable-lv2 --enable-openssl --enable-pic --enable-postproc --enable-pthreads --enable-shared --enable-vaapi --enable-vdpau --enable-version3 --enable-vulkan --optflags=-O3 --enable-libjxl --enable-libsvtav1 --enable-libvpl
  libavutil      58. 29.100 / 58. 29.100
  libavcodec     60. 31.102 / 60. 31.102
  libavformat    60. 16.100 / 60. 16.100
  libavdevice    60.  3.100 / 60.  3.100
  libavfilter     9. 12.100 /  9. 12.100
  libswscale      7.  5.100 /  7.  5.100
  libswresample   5.  0.100 /  5.  0.100
  libpostproc    57.  3.100 / 57.  3.100
Input #0, rtsp, from 'rtsp:https://192.168.0.110:554/11':
  Metadata:
    title           : 10
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: hevc (Main), yuv420p(tv, bt709), 1920x1080, 20 fps, 20 tbr, 90k tbn
  Stream #0:1: Audio: pcm_alaw, 8000 Hz, mono, s16, 64 kb/s

Frigate stats

No response

Operating system

Other Linux

Install method

Docker Compose

Coral version

CPU (no coral)

Network connection

Wireless

Camera make and model

Topcony

Any other information that may be helpful

No response

[NickM-27]

How are you assessing that ptz is slow? You're not using go2rtc which means you're using jsmpeg for live view which has high latency, but that doesn't mean that ptz itself is slow.

[littlebritaly]

Hi,
Thanks for the reply.
I am indeed using jsmpeg for live view, at the moment i have no idea what go2rtc is, but will now check it out and implement it (Thank you).

When I say slow, what I mean is it is slow to react, I press a button and nothing seems to happen or if it does it is seconds later.

With my Security Spy setup, whe i press a button, movement happens more or less straight away.

I will report back with results.

[NickM-27]

then that is because you are using jsmpeg which has high latency. Webrtc is best for this type of action

https://docs.frigate.video/guides/configuring_go2rtc