Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Support]: Exception in thread Thread-12/13/14 #7257

Open
LckySndays opened this issue Jul 23, 2023 · 24 comments
Open

[Support]: Exception in thread Thread-12/13/14 #7257

LckySndays opened this issue Jul 23, 2023 · 24 comments
Labels
bug Something isn't working pinned

Comments

@LckySndays
Copy link

Describe the problem you are having

Roughly for the last month, my frigate crashed with the following Exception in thread Thread-12/13/14 almost once everyday. This error cause the Debug view for all camera not loading up but the Live view is working properly. Anyone know what may cause this issue?

Version

0.13.0-D0413E3

Frigate config file

mqtt:
  enabled: true
  host: 192.168.100.10
  port: 1883
  user: xxx
  password: yyy

birdseye:
  enabled: true
  #restream: true
  width: 1920
  height: 1080
  quality: 1
  mode: objects

live:
  quality: 1
  
detect:
  enabled: true
  fps: 10
  max_disappeared: 50
  stationary:
    interval: 50
    threshold: 100

detectors:
  #CPU:
  #  type: cpu
  Coral1:
    type: edgetpu
    device: pci:0
  Coral2:
   type: edgetpu
   device: pci:1

objects:
  track:
    - person
  #filters:
  #  person:
  #    threshold: 0.75
  #  dog:
  #    threshold: 0.75

snapshots:
  enabled: true
  clean_copy: true
  timestamp: false
  bounding_box: false
  crop: false
  retain:
    default: 2

ui:
  live_mode: webrtc
  timezone: Asia/Taipei
  #use_experimental: true

telemetry:
  version_check: false
    
ffmpeg:
  hwaccel_args: preset-nvidia-h264
  input_args: preset-rtsp-restream-low-latency

database:
  path: /config/frigate.db

go2rtc:
  streams:
    CAM1-01: rtsp:https://xxx:[email protected]/Streaming/Channels/101
    CAM1-01-LRes: ffmpeg:rtsp:https://xxx:[email protected]/Streaming/Channels/102#video=copy#audio=aac#audio=opus#async#hardware#input=rtsp/udp
    CAM1-02: rtsp:https://xxx:[email protected]/Streaming/Channels/101
    CAM1-02-LRes: ffmpeg:rtsp:https://xxx:[email protected]/Streaming/Channels/102#video=copy#audio=aac#audio=opus#async#hardware#input=rtsp/udp
    CAM2: rtsp:https://xxx:[email protected]/Streaming/Channels/101
    CAM2-LRes: ffmpeg:rtsp:https://xxx:[email protected]/Streaming/Channels/102#video=copy#audio=aac#audio=opus#async#hardware#input=rtsp/udp
    CAM3: rtsp:https://xxx:[email protected]/Streaming/Channels/101
    CAM3-LRes: ffmpeg:rtsp:https://xxx:[email protected]/Streaming/Channels/102#video=copy#audio=aac#audio=opus#async#hardware#input=rtsp/udp
    CAM4: rtsp:https://xxx:[email protected]/Streaming/Channels/101
    CAM4-LRes: ffmpeg:rtsp:https://xxx:[email protected]/Streaming/Channels/102#video=copy#audio=aac#audio=opus#async#hardware#input=rtsp/udp
    CAM5: rtsp:https://xxx:[email protected]/Streaming/Channels/101
    CAM5-LRes: ffmpeg:rtsp:https://xxx:[email protected]/Streaming/Channels/102#video=copy#audio=aac#audio=opus#async#hardware#input=rtsp/udp
    CAM6: rtsp:https://xxx:[email protected]/Streaming/Channels/101
    CAM6-LRes: ffmpeg:rtsp:https://xxx:[email protected]/Streaming/Channels/102#video=copy#audio=aac#audio=opus#async#hardware#input=rtsp/udp
    CAM7: rtsp:https://xxx:[email protected]/Streaming/Channels/101
    CAM7-LRes: ffmpeg:rtsp:https://xxx:[email protected]/Streaming/Channels/102#video=copy#audio=aac#audio=opus#async#hardware#input=rtsp/udp
    CAM8: rtsp:https://xxx:[email protected]/Streaming/Channels/101
    CAM8-LRes: ffmpeg:rtsp:https://xxx:[email protected]/Streaming/Channels/102#video=copy#audio=aac#audio=opus#async#hardware#input=rtsp/udp
    CAM9: rtsp:https://xxx:[email protected]/Streaming/Channels/101
    CAM9-LRes: ffmpeg:rtsp:https://xxx:[email protected]/Streaming/Channels/102#video=copy#audio=aac#audio=opus#async#hardware#input=rtsp/udp
    CAM8-02: rtsp:https://xxx:[email protected]/Streaming/Channels/101
    CAM8-02-LRes: ffmpeg:rtsp:https://xxx:[email protected]/Streaming/Channels/102#video=copy#audio=aac#audio=opus#async#hardware#input=rtsp/udp
    CAM10: rtsp:https://xxx:[email protected]/Streaming/Channels/101
    CAM10-LRes: ffmpeg:rtsp:https://xxx:[email protected]/Streaming/Channels/102#video=copy#audio=aac#audio=opus#async#hardware#input=rtsp/udp
    CAM11: rtsp:https://xxx:[email protected]/Streaming/Channels/101
    CAM11-LRes: ffmpeg:rtsp:https://xxx:[email protected]/Streaming/Channels/102#video=copy#audio=aac#audio=opus#async#hardware#input=rtsp/udp
    CAM12: rtsp:https://xxx:[email protected]/Streaming/Channels/101
    CAM12-LRes: ffmpeg:rtsp:https://xxx:[email protected]/Streaming/Channels/102#video=copy#audio=aac#audio=opus#async#hardware#input=rtsp/udp
    CAM13-01: rtsp:https://xxx:[email protected]:554/Streaming/Channels/101?transportmode=unicast&profile=Profile_1
    CAM13-01-LRes: ffmpeg:rtsp:https://xxx:[email protected]:554/Streaming/Channels/102?transportmode=unicast&profile=Profile_2#video=copy#audio=pcma#audio=aac#audio=opus#async#hardware#input=rtsp/udp
    #CAM13-01: rtsp:https://xxx:[email protected]/Streaming/Channels/101
    #CAM13-01-LRes: ffmpeg:rtsp:https://xxx:[email protected]/Streaming/Channels/102#video=copy#audio=aac#audio=opus#async#hardware#input=rtsp/udp
    CAM13-02: rtsp:https://xxx:[email protected]/Streaming/Channels/101
    CAM13-02-LRes: ffmpeg:rtsp:https://xxx:[email protected]/Streaming/Channels/102#video=copy#audio=aac#audio=opus#async#hardware#input=rtsp/udp
    #CAM114: rtsp:https://xxx:[email protected]/Streaming/Channels/101
    CAM114: ffmpeg:rtsp:https://xxx:[email protected]/Streaming/Channels/101#video=copy#audio=aac#audio=opus#async#hardware#input=rtsp/udp
    #CAM114-LRes: ffmpeg:rtsp:https://xxx:[email protected]/Streaming/Channels/102#video=copy#audio=aac#audio=opus#async#hardware#input=rtsp/udp
    TestStream:

  webrtc:
    candidates:
      - 192.168.100.10:8555
      - stun:8555
  
  rtsp:
    listen: ":8554"    # RTSP Server TCP port, default - 8554
    username: "xxx"  # optional, default - disabled
    password: "yyy"   # optional, default - disabled
    default_query: "video&audio"  # optional, default codecs filters 

  log:
    #level: trace
    level: error
      
cameras:
  CAM1-01:
    ffmpeg:
      inputs:
        - path: rtsp:https://localhost:8554/CAM1-01
    #birdseye:
    #  mode: objects
    live:
      stream_name: CAM1-01-LRes
      #height: 2160
    detect:
      width: 3840
      height: 2160
    #motion:
    #  threshold: 30
    #  mask:
    #    - 3840,0,3840,760,2124,1555,0,2160,0,888,0,0
    objects:
      track:
        - person
        - car
        - bicycle
        - motorcycle
        #- bird
        #- cat
        - dog
      filters:
        person:
          min_area: 15000
          max_area: 265000
          min_score: 0.75
        car:
          min_area: 65000
          max_area: 1800000
          min_score: 0.75
        bicycle:
          min_area: 40000
          max_area: 230000
          min_score: 0.75
        motorcycle:
          min_area: 40000
          max_area: 230000
          min_score: 0.75
        dog:
          min_area: 10000
          max_area: 120000
          min_score: 0.75
        
  CAM1-02:
    ffmpeg:
      inputs:
        - path: rtsp:https://localhost:8554/CAM1-02
    #birdseye:
    #  mode: objects
    live:
      stream_name: CAM1-02-LRes
      #height: 2160
    detect:
      width: 3840
      height: 2160
    #motion:
    #  threshold: 30
    #  mask:
    #    - 3840,0,3840,760,2124,1555,0,2160,0,888,0,0
    objects:
      track:
        - person
        - car
        - bicycle
        - motorcycle
        #- bird
        #- cat
        - dog
      filters:
        person:
          min_area: 20000
          max_area: 230000
          min_score: 0.75
        car:
          min_area: 100000
          max_area: 900000
          min_score: 0.75
        bicycle:
          min_area: 40000
          max_area: 250000
          min_score: 0.75
        motorcycle:
          min_area: 40000
          max_area: 250000
          min_score: 0.75
        dog:
          min_area: 10000
          max_area: 95000
          min_score: 0.75

  CAM2:
    ffmpeg:
      inputs:
        - path: rtsp:https://localhost:8554/CAM2
    #birdseye:
    #  mode: objects
    live:
      stream_name: CAM2-LRes
      #height: 2160
    detect:
      width: 3840
      height: 2160
    #motion:
      #threshold: 50
      #contour_area: 180
      #frame_height: 150
    objects:
      track:
        - person
        - car
        - bicycle
        - motorcycle
        #- bird
        #- cat
        - dog
      filters:
        person:
          min_area: 10000
          max_area: 500000
          min_score: 0.75
        car:
          min_area: 100000
          max_area: 2250000
          min_score: 0.75
        bicycle:
          min_area: 25000
          max_area: 75000
          min_score: 0.75
        motorcycle:
          min_area: 25000
          max_area: 75000
          min_score: 0.75
          mask:
            - 2798,557,2782,898,1964,726,1959,520 # motorcycle parking lot
        dog:
          min_area: 8000
          max_area: 140000
          min_score: 0.75
    zones:
      CAM2-Gate:
        coordinates: 3840,891,3840,1716,2624,1204,2657,591 # 2023/07/03
        objects:
          - car
        filters:
          car:
           min_area: 100000
           max_area: 2250000
           min_score: 0.75

  CAM3:
    ffmpeg:
      inputs:
        - path: rtsp:https://localhost:8554/CAM3
    live:
      stream_name: CAM3-LRes
      #height: 1800
    detect:
      width: 3200
      height: 1800
    objects:
      filters:
        person:
          min_area: 50000
          max_area: 350000
          min_score: 0.7

  CAM4:
    ffmpeg:
      hwaccel_args: preset-vaapi
      inputs:
        - path: rtsp:https://localhost:8554/CAM4
    live:
      stream_name: CAM4-LRes
      #height: 1440
    detect:
      width: 2560
      height: 1440
    #motion:
    #  threshold: 30
    #objects:
    #  filters:
    #    person:
    #      threshold: 0.75
      
  CAM5:
    ffmpeg:
      inputs:
        - path: rtsp:https://localhost:8554/CAM5
    #birdseye:
    #  mode: continuous
    live:
      stream_name: CAM5-LRes
      #height: 2160
    detect:
      width: 3840
      height: 2160
    #motion:
    #  threshold: 30
    #  contour_area: 180
    #  frame_height: 150
    #  mask:
    #    - 362,1264,610,1201,512,989,303,1083,95,618,1193,441,2540,575,3840,882,3840,0,0,0,0,2160,1032,2160 # unnecessary frame
    objects:
      track:
        - person
        - car
        - bicycle
        - motorcycle
        #- bird
        #- cat
        - dog
      filters:
        person:
          min_area: 10500
          max_area: 150000
          min_score: 0.75
        car:
          min_area: 75000
          max_area: 2400000
          min_score: 0.75
        bicycle:
          min_area: 15000
          max_area: 400000
          min_score: 0.75
        motorcycle:
          min_area: 15000
          max_area: 400000
          min_score: 0.75
          mask:
            - 0,1570,726,1554,733,699,0,708 # motorcycle parking lot
        dog:
          min_area: 7000
          max_area: 140000
          min_score: 0.75
      #mask:
      #  - 2020,1048,2355,819,2209,670,1757,910 # false detection birdview active object
       
  CAM6:
    ffmpeg:
      inputs:
        - path: rtsp:https://localhost:8554/CAM6
    #birdseye:
    #  mode: motion
    live:
      stream_name: CAM6-LRes
      #height: 2160
    detect:
      width: 3840
      height: 2160
  
  CAM7:
    ffmpeg:
      inputs:
        - path: rtsp:https://localhost:8554/CAM7
    #birdseye:
    #  mode: motion
    live:
      stream_name: CAM7-LRes
      #height: 2160
    detect:
      width: 3200
      height: 1800
       
  CAM8:
    ffmpeg:
      inputs:
        - path: rtsp:https://localhost:8554/CAM8
    #birdseye:
    #  mode: motion
    live:
      stream_name: CAM8-LRes
      #height: 1800
    detect:
      width: 3200
      height: 1800
       
  CAM9:
    ffmpeg:
      inputs:
        - path: rtsp:https://localhost:8554/CAM9
    #birdseye:
    #  mode: objects
    live:
      stream_name: CAM9-LRes
      #height: 1800
    detect:
      width: 3200
      height: 1800
    #motion:
    #  threshold: 50 # too many motion detected
    #  mask:
    #    - 0,1800,845,1800,403,1004,1414,309,2689,1009,2297,1800,3200,1800,3200,0,0,0 # unnecessary frame
    objects:
      filters:
        person:
          min_area: 70000
          max_area: 300000
          min_score: 0.6
          mask:
            - 568,1574,1434,863,1154,660,333,1244
            - 1599,1646,1260,1358,1770,885,2094,1090
  
  CAM8-02:
    ffmpeg:
      inputs:
        - path: rtsp:https://localhost:8554/CAM8-02
    #birdseye:
    #  mode: motion
    live:
      stream_name: CAM8-02-LRes
      #height: 1800
    detect:
      width: 3200
      height: 1800
  
  CAM10:
    ffmpeg:
      inputs:
        - path: rtsp:https://localhost:8554/CAM10
    live:
      stream_name: CAM10-LRes
    detect:
      width: 3840
      height: 2160
    objects:
      track:
        - person
      filters:
        person:
          min_area: 20000
          max_area: 300000
          min_score: 0.75

  CAM11:
    ffmpeg:
      inputs:
        - path: rtsp:https://localhost:8554/CAM11
    #birdseye:
    #  mode: objects
    live:
      stream_name: CAM11-LRes
      #height: 2160
    detect:
      width: 3840
      height: 2160
    #motion:
    #  threshold: 30
    #  contour_area: 180
    #  frame_height: 150
    #  mask:
    #   - 0,877,837,612,971,498,2139,390,2076,0,0,0 # unnecessary frame
    objects:
      track:
        - person
        #- bird
      filters:
        person:
          min_area: 20000
          max_area: 300000
          min_score: 0.75
      #mask:
      #  - 870,2160,1331,2160,1343,1709,863,1705 # false detection birdview active object
       
  CAM12:
    ffmpeg:
      inputs:
        - path: rtsp:https://localhost:8554/CAM12
    live:
      stream_name: CAM12-LRes
      #height: 1800
    detect:
      width: 3200
      height: 1800
    objects:
      filters:
        person:
          min_area: 25000
          max_area: 660000
          min_score: 0.6
       
  CAM13-01:
    ffmpeg:
      inputs:
        - path: rtsp:https://localhost:8554/CAM13-01
    live:
      stream_name: CAM13-01-LRes
      #height: 2160
    detect:
      width: 3840
      height: 2160
    #motion:
      #threshold: 30
    objects:
      filters:
        person:
          min_area: 30000
          max_area: 90000
          min_score: 0.6
          mask:
            - 1603,75,1697,504,1075,1359,839,1284,634,953 #stocker
       
  CAM13-02:
    ffmpeg:
      inputs:
        - path: rtsp:https://localhost:8554/CAM13-02
    live:
      stream_name: CAM13-02-LRes
      #height: 2160
    detect:
      width: 3840
      height: 2160
    #motion:
    #  mask:
    #    - 933,437,3840,886,3840,0,2666,0,1875,142
    objects:
      filters:
        person:
          min_area: 20000
          max_area: 70000
          min_score: 0.6
          mask:
            - 3840,442,3840,1432,3330,1437,3355,445 # stocker
  
  CAM114:
    enabled: false
    ffmpeg:
      inputs:
        - path: rtsp:https://localhost:8554/CAM114
    #live:
    #  stream_name: CAM114-LRes
      #height: 2160
    record:
      enabled: True
      retain:
        days: 7
        mode: all
    detect:
      width: 3200
      #width: 1280
      height: 1800
      #height: 720

Relevant log output

2023-07-09 20:37:23.479838841  Exception in thread Thread-12:
2023-07-09 20:37:23.479850717  Traceback (most recent call last):
2023-07-09 20:37:23.479860086    File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
2023-07-09 20:37:23.480961510      self.run()
2023-07-09 20:37:23.480966854    File "/opt/frigate/frigate/output.py", line 151, in run
2023-07-09 20:37:23.481209196      ws.send(buf, binary=True)
2023-07-09 20:37:23.481214234    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 303, in send
2023-07-09 20:37:23.481284497      self._write(m)
2023-07-09 20:37:23.481287199    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 285, in _write
2023-07-09 20:37:23.481331593  [2023-07-09 20:37:23] ws4py                          ERROR   : Failed to receive data
2023-07-09 20:37:23.481332199  Traceback (most recent call last):
2023-07-09 20:37:23.481332982    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 394, in once
2023-07-09 20:37:23.481333779      b = self.sock.recv(self.reading_buffer_size)
2023-07-09 20:37:23.481334364  ConnectionResetError: [Errno 104] Connection reset by peer
2023-07-09 20:37:23.481335018      self.sock.sendall(b)
2023-07-09 20:37:23.481520701  BrokenPipeError: [Errno 32] Broken pipe

2023-07-14 10:20:25.363788788  Exception in thread Thread-12:
2023-07-14 10:20:25.363799522  Traceback (most recent call last):
2023-07-14 10:20:25.363806411    File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
2023-07-14 10:20:25.364809993  [2023-07-14 10:20:25] ws4py                          ERROR   : Failed to receive data
2023-07-14 10:20:25.364810756  Traceback (most recent call last):
2023-07-14 10:20:25.364811313    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 394, in once
2023-07-14 10:20:25.364811863      b = self.sock.recv(self.reading_buffer_size)
2023-07-14 10:20:25.364812400  ConnectionResetError: [Errno 104] Connection reset by peer
2023-07-14 10:20:25.364883748      self.run()
2023-07-14 10:20:25.364887551    File "/opt/frigate/frigate/output.py", line 210, in run
2023-07-14 10:20:25.365075860      ws.send(buf, binary=True)
2023-07-14 10:20:25.365079820    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 303, in send
2023-07-14 10:20:25.365129504      self._write(m)
2023-07-14 10:20:25.365131710    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 285, in _write
2023-07-14 10:20:25.365172509      self.sock.sendall(b)
2023-07-14 10:20:25.365186172  BrokenPipeError: [Errno 32] Broken pipe

2023-07-15 19:32:07.298251328  Exception in thread Thread-14:
2023-07-15 19:32:07.298261015  Traceback (most recent call last):
2023-07-15 19:32:07.298269054    File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
2023-07-15 19:32:07.299050363      self.run()
2023-07-15 19:32:07.299053011    File "/opt/frigate/frigate/output.py", line 210, in run
2023-07-15 19:32:07.299317602      ws.send(buf, binary=True)
2023-07-15 19:32:07.299318568    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 303, in send
2023-07-15 19:32:07.299429248  [2023-07-15 19:32:07] ws4py                          ERROR   : Failed to receive data
2023-07-15 19:32:07.299429963  Traceback (most recent call last):
2023-07-15 19:32:07.299430868    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 394, in once
2023-07-15 19:32:07.299431622      b = self.sock.recv(self.reading_buffer_size)
2023-07-15 19:32:07.299432466  ConnectionResetError: [Errno 104] Connection reset by peer
2023-07-15 19:32:07.299433111      self._write(m)
2023-07-15 19:32:07.299437147    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 285, in _write
2023-07-15 19:32:07.299452445      self.sock.sendall(b)
2023-07-15 19:32:07.299461104  BrokenPipeError: [Errno 32] Broken pipe

2023-07-18 20:02:40.781424602  Exception in thread Thread-13:
2023-07-18 20:02:40.781432187  Traceback (most recent call last):
2023-07-18 20:02:40.781432838    File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
2023-07-18 20:02:40.781898270      self.run()
2023-07-18 20:02:40.781898796    File "/opt/frigate/frigate/output.py", line 210, in run
2023-07-18 20:02:40.781924257      ws.send(buf, binary=True)
2023-07-18 20:02:40.781924832    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 303, in send
2023-07-18 20:02:40.781955264      self._write(m)
2023-07-18 20:02:40.781955821    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 285, in _write

2023-07-20 08:50:13.723632484  Exception in thread Thread-14:
2023-07-20 08:50:13.723641826  Traceback (most recent call last):
2023-07-20 08:50:13.723642719    File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
2023-07-20 08:50:13.724371870      self.run()
2023-07-20 08:50:13.724372709    File "/opt/frigate/frigate/output.py", line 210, in run
2023-07-20 08:50:13.724633564      ws.send(buf, binary=True)
2023-07-20 08:50:13.724634337    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 303, in send
2023-07-20 08:50:13.724842783      self._write(m)
2023-07-20 08:50:13.724843558    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 285, in _write
2023-07-20 08:50:13.724862834      self.sock.sendall(b)
2023-07-20 08:50:13.724924782  ConnectionResetError: [Errno 104] Connection reset by peer

2023-07-22 20:10:44.377621216  Exception in thread Thread-14:
2023-07-22 20:10:44.377656385  Traceback (most recent call last):
2023-07-22 20:10:44.377660373    File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
2023-07-22 20:10:44.378621043  [2023-07-22 20:10:44] ws4py                          ERROR   : Failed to receive data
2023-07-22 20:10:44.378621817  Traceback (most recent call last):
2023-07-22 20:10:44.378622621    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 394, in once
2023-07-22 20:10:44.378623335      b = self.sock.recv(self.reading_buffer_size)
2023-07-22 20:10:44.378624027  ConnectionResetError: [Errno 104] Connection reset by peer
2023-07-22 20:10:44.378757086      self.run()
2023-07-22 20:10:44.378760250    File "/opt/frigate/frigate/output.py", line 210, in run
2023-07-22 20:10:44.378805820      ws.send(buf, binary=True)
2023-07-22 20:10:44.378808838    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 303, in send
2023-07-22 20:10:44.378852342      self._write(m)
2023-07-22 20:10:44.378855433    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 285, in _write
2023-07-22 20:10:44.378891651      self.sock.sendall(b)
2023-07-22 20:10:44.378904652  BrokenPipeError: [Errno 32] Broken pipe

FFprobe output from your camera

[{"return_code":0,"stderr":"","stdout":{"programs":[],"streams":[{"avg_frame_rate":"20/1","codec_long_name":"H.264/AVC/MPEG-4AVC/MPEG-4part10","height":2160,"width":3840},{"avg_frame_rate":"0/0","codec_long_name":"AAC(AdvancedAudioCoding)"}]}}]

Frigate stats

{"CAM4":{"camera_fps":10.1,"capture_pid":1414,"detection_enabled":1,"detection_fps":2.3,"ffmpeg_pid":1421,"pid":1360,"process_fps":10.1,"skipped_fps":0.0},"CAM3":{"camera_fps":10.1,"capture_pid":1409,"detection_enabled":1,"detection_fps":0.0,"ffmpeg_pid":1415,"pid":1357,"process_fps":10.0,"skipped_fps":0.0},"CAM1-01":{"camera_fps":10.0,"capture_pid":1393,"detection_enabled":1,"detection_fps":1.6,"ffmpeg_pid":1399,"pid":1353,"process_fps":9.9,"skipped_fps":0.0},"CAM1-02":{"camera_fps":10.1,"capture_pid":1397,"detection_enabled":1,"detection_fps":22.0,"ffmpeg_pid":1404,"pid":1354,"process_fps":10.1,"skipped_fps":0.0},"CAM2":{"camera_fps":10.0,"capture_pid":1403,"detection_enabled":1,"detection_fps":29.5,"ffmpeg_pid":1410,"pid":1356,"process_fps":10.1,"skipped_fps":0.0},"CAM9":{"camera_fps":10.0,"capture_pid":1442,"detection_enabled":1,"detection_fps":2.7,"ffmpeg_pid":1449,"pid":1374,"process_fps":9.9,"skipped_fps":0.0},"CAM7":{"camera_fps":10.0,"capture_pid":1431,"detection_enabled":1,"detection_fps":0.0,"ffmpeg_pid":1438,"pid":1369,"process_fps":9.8,"skipped_fps":0.0},"CAM6":{"camera_fps":10.0,"capture_pid":1425,"detection_enabled":1,"detection_fps":0.0,"ffmpeg_pid":1432,"pid":1367,"process_fps":9.8,"skipped_fps":0.0},"CAM5":{"camera_fps":10.1,"capture_pid":1420,"detection_enabled":1,"detection_fps":38.3,"ffmpeg_pid":1427,"pid":1362,"process_fps":10.0,"skipped_fps":0.0},"CAM8":{"camera_fps":10.1,"capture_pid":1437,"detection_enabled":1,"detection_fps":0.8,"ffmpeg_pid":1444,"pid":1371,"process_fps":9.8,"skipped_fps":0.0},"CAM8-02":{"camera_fps":10.1,"capture_pid":1448,"detection_enabled":1,"detection_fps":0.7,"ffmpeg_pid":1456,"pid":1378,"process_fps":10.1,"skipped_fps":0.0},"CAM10":{"camera_fps":10.0,"capture_pid":1454,"detection_enabled":1,"detection_fps":0.0,"ffmpeg_pid":1480,"pid":1379,"process_fps":10.0,"skipped_fps":0.0},"CAM13":{"camera_fps":10.1,"capture_pid":1488,"detection_enabled":1,"detection_fps":0.0,"ffmpeg_pid":1494,"pid":1387,"process_fps":10.1,"skipped_fps":0.0},"CAM14":{"camera_fps":10.1,"capture_pid":1493,"detection_enabled":1,"detection_fps":0.6,"ffmpeg_pid":1500,"pid":1390,"process_fps":9.9,"skipped_fps":0.0},"CAM12":{"camera_fps":10.2,"capture_pid":1483,"detection_enabled":1,"detection_fps":0.8,"ffmpeg_pid":1490,"pid":1385,"process_fps":9.9,"skipped_fps":0.0},"CAM11":{"camera_fps":10.0,"capture_pid":1478,"detection_enabled":1,"detection_fps":0.1,"ffmpeg_pid":1484,"pid":1382,"process_fps":9.9,"skipped_fps":0.0},"CAM15":{"camera_fps":0.0,"capture_pid":null,"detection_enabled":1,"detection_fps":0.0,"ffmpeg_pid":0,"pid":null,"process_fps":0.0,"skipped_fps":0.0},"cpu_usages":{"1":{"cmdline":"/package/admin/s6/command/s6-svscan -d4 -- /run/service","cpu":"0.0","cpu_average":"0","mem":"0.0"},"14":{"cmdline":"s6-supervise s6-linux-init-shutdownd","cpu":"0.0","cpu_average":"0","mem":"0.0"},"15":{"cmdline":"/package/admin/s6-linux-init/command/s6-linux-init-shutdownd -c /run/s6/basedir -g 3000 -C -B","cpu":"0.0","cpu_average":"0","mem":"0.0"},"23":{"cmdline":"s6-supervise go2rtc-healthcheck","cpu":"0.0","cpu_average":"0","mem":"0.0"},"24":{"cmdline":"s6-supervise go2rtc-log","cpu":"0.0","cpu_average":"0","mem":"0.0"},"25":{"cmdline":"s6-supervise s6rc-oneshot-runner","cpu":"0.0","cpu_average":"0","mem":"0.0"},"26":{"cmdline":"s6-supervise go2rtc","cpu":"0.0","cpu_average":"0","mem":"0.0"},"27":{"cmdline":"s6-supervise frigate","cpu":"0.0","cpu_average":"0","mem":"0.0"},"28":{"cmdline":"s6-supervise nginx-log","cpu":"0.0","cpu_average":"0","mem":"0.0"},"29":{"cmdline":"s6-supervise nginx","cpu":"0.0","cpu_average":"0","mem":"0.0"},"30":{"cmdline":"s6-supervise frigate-log","cpu":"0.0","cpu_average":"0","mem":"0.0"},"31":{"cmdline":"s6-supervise s6rc-fdholder","cpu":"0.0","cpu_average":"0","mem":"0.0"},"40":{"cmdline":"/package/admin/s6-2.11.3.0/command/s6-fdholderd -1 -i data/rules","cpu":"0.0","cpu_average":"0","mem":"0.0"},"41":{"cmdline":"/package/admin/s6/command/s6-ipcserverd -1 -- /package/admin/s6/command/s6-ipcserver-access -v0 -E -l0 -i data/rules -- /package/admin/s6/command/s6-sudod -t 30000 -- /package/admin/s6-rc/command/s6-rc-oneshot-run -l ../.. --","cpu":"0.0","cpu_average":"0","mem":"0.0"},"79":{"cmdline":"s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/nginx","cpu":"0.0","cpu_average":"0","mem":"0.0"},"80":{"cmdline":"s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/go2rtc","cpu":"0.0","cpu_average":"0","mem":"0.0"},"81":{"cmdline":"s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/frigate","cpu":"0.0","cpu_average":"0","mem":"0.0"},"88":{"cmdline":"/config/go2rtc -config=/dev/shm/go2rtc.yaml","cpu":"15.6","cpu_average":"13","mem":"0.0"},"96":{"cmdline":"bash ./run.user go2rtc-healthcheck","cpu":"0.0","cpu_average":"0","mem":"0.0"},"97":{"cmdline":"python3 -u -m frigate","cpu":"20.3","cpu_average":"10","mem":"2.7"},"113":{"cmdline":"nginx: master process nginx","cpu":"0.0","cpu_average":"0","mem":"0.0"},"143":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"144":{"cmdline":"nginx: worker process","cpu":"0.1","cpu_average":"0","mem":"0.0"},"145":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"146":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"152":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"165":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"184":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"213":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"244":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"273":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"297":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"318":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"340":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"367":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"394":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"428":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"460":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"488":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"513":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"540":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"582":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"628":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"679":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"724":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"760":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"790":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"823":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"863":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"920":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"960":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"999":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"1032":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"1302":{"cmdline":"frigate.logger       ","cpu":"0.0","cpu_average":"0","mem":"0.2"},"1304":{"cmdline":"frigate.recording_manager","cpu":"0.2","cpu_average":"0","mem":"0.4"},"1313":{"cmdline":"/usr/bin/python3 -c from multiprocessing.resource_tracker import main;main(71)","cpu":"0.7","cpu_average":"0","mem":"0.0"},"1314":{"cmdline":"frigate.detector.Coral1","cpu":"9.1","cpu_average":"3","mem":"0.4"},"1316":{"cmdline":"frigate.detector.Coral2","cpu":"9.3","cpu_average":"3","mem":"0.4"},"1320":{"cmdline":"frigate.output       ","cpu":"15.9","cpu_average":"14","mem":"0.5"},"1352":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 3840x2160 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 1 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"1353":{"cmdline":"frigate.process:CAM1-01","cpu":"9.8","cpu_average":"6","mem":"0.5"},"1354":{"cmdline":"frigate.process:CAM1-02","cpu":"35.5","cpu_average":"9","mem":"0.5"},"1355":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 3840x2160 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 1 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"1356":{"cmdline":"frigate.process:CAM2","cpu":"133.1","cpu_average":"16","mem":"0.5"},"1357":{"cmdline":"frigate.process:CAM3","cpu":"2.9","cpu_average":"9","mem":"0.5"},"1359":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 3840x2160 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 1 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"1360":{"cmdline":"frigate.process:CAM4","cpu":"3.6","cpu_average":"14","mem":"0.5"},"1362":{"cmdline":"frigate.process:CAM5","cpu":"106.5","cpu_average":"11","mem":"0.5"},"1363":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 3200x1800 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 1 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"1367":{"cmdline":"frigate.process:CAM6","cpu":"1.9","cpu_average":"4","mem":"0.5"},"1368":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 2560x1440 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 1 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"1369":{"cmdline":"frigate.process:CAM7","cpu":"1.3","cpu_average":"3","mem":"0.5"},"1371":{"cmdline":"frigate.process:CAM8","cpu":"3.1","cpu_average":"5","mem":"0.5"},"1372":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 3840x2160 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 1 -bf 0 pipe:","cpu":"18.2","cpu_average":"16","mem":"0.1"},"1374":{"cmdline":"frigate.process:CAM9","cpu":"4.8","cpu_average":"6","mem":"0.4"},"1377":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 3840x2160 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 1 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"1378":{"cmdline":"frigate.process:CAM8-02","cpu":"1.6","cpu_average":"3","mem":"0.4"},"1379":{"cmdline":"frigate.process:CAM10","cpu":"5.5","cpu_average":"2","mem":"0.5"},"1381":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 3200x1800 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 1 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"1382":{"cmdline":"frigate.process:CAM11","cpu":"1.7","cpu_average":"3","mem":"0.5"},"1384":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 3200x1800 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 1 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"1385":{"cmdline":"frigate.process:CAM12","cpu":"2.9","cpu_average":"3","mem":"0.4"},"1387":{"cmdline":"frigate.process:CAM13","cpu":"11.9","cpu_average":"4","mem":"0.4"},"1388":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 3200x1800 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 1 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"1390":{"cmdline":"frigate.process:CAM14","cpu":"11.5","cpu_average":"10","mem":"0.5"},"1392":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 3200x1800 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 1 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"1393":{"cmdline":"frigate.capture:CAM1-01","cpu":"9.7","cpu_average":"8","mem":"0.4"},"1397":{"cmdline":"frigate.capture:CAM1-02","cpu":"9.2","cpu_average":"7","mem":"0.4"},"1398":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 3840x2160 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 1 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"1399":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.13.0-d0413e3 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp:https://localhost:8554/CAM1-01 -r 10 -vf fps=10,scale_cuda=w=3840:h=2160:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"11.0","cpu_average":"9","mem":"0.6"},"1403":{"cmdline":"frigate.capture:CAM2","cpu":"9.0","cpu_average":"7","mem":"0.4"},"1404":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.13.0-d0413e3 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp:https://localhost:8554/CAM1-02 -r 10 -vf fps=10,scale_cuda=w=3840:h=2160:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"16.3","cpu_average":"13","mem":"0.5"},"1408":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 3840x2160 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 1 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"1409":{"cmdline":"frigate.capture:CAM3","cpu":"6.8","cpu_average":"5","mem":"0.4"},"1410":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.13.0-d0413e3 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp:https://localhost:8554/CAM2 -r 10 -vf fps=10,scale_cuda=w=3840:h=2160:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"15.6","cpu_average":"13","mem":"0.5"},"1414":{"cmdline":"frigate.capture:CAM4","cpu":"4.4","cpu_average":"4","mem":"0.4"},"1415":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.13.0-d0413e3 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp:https://localhost:8554/CAM3 -r 10 -vf fps=10,scale_cuda=w=3200:h=1800:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"8.5","cpu_average":"6","mem":"0.5"},"1419":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 3200x1800 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 1 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"1420":{"cmdline":"frigate.capture:CAM5","cpu":"7.7","cpu_average":"7","mem":"0.4"},"1421":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.0-d0413e3 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp:https://localhost:8554/CAM4 -r 10 -vf fps=10,scale_vaapi=w=2560:h=1440,hwdownload,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"7.9","cpu_average":"6","mem":"0.1"},"1425":{"cmdline":"frigate.capture:CAM6","cpu":"9.0","cpu_average":"7","mem":"0.4"},"1427":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.13.0-d0413e3 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp:https://localhost:8554/CAM5 -r 10 -vf fps=10,scale_cuda=w=3840:h=2160:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"14.4","cpu_average":"12","mem":"0.5"},"1428":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 3840x2160 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 1 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"1431":{"cmdline":"frigate.capture:CAM7","cpu":"5.9","cpu_average":"5","mem":"0.4"},"1432":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.13.0-d0413e3 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp:https://localhost:8554/CAM6 -r 10 -vf fps=10,scale_cuda=w=3840:h=2160:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"14.6","cpu_average":"13","mem":"0.4"},"1436":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 3840x2160 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 1 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"1437":{"cmdline":"frigate.capture:CAM8","cpu":"6.6","cpu_average":"5","mem":"0.4"},"1438":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.13.0-d0413e3 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp:https://localhost:8554/CAM7 -r 10 -vf fps=10,scale_cuda=w=3200:h=1800:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"6.7","cpu_average":"6","mem":"0.5"},"1442":{"cmdline":"frigate.capture:CAM9","cpu":"6.7","cpu_average":"5","mem":"0.4"},"1444":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.13.0-d0413e3 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp:https://localhost:8554/CAM8 -r 10 -vf fps=10,scale_cuda=w=3200:h=1800:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"7.9","cpu_average":"6","mem":"0.5"},"1445":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 3200x1800 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 1 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"1448":{"cmdline":"frigate.capture:CAM8-02","cpu":"6.4","cpu_average":"5","mem":"0.4"},"1449":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.13.0-d0413e3 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp:https://localhost:8554/CAM9 -r 10 -vf fps=10,scale_cuda=w=3200:h=1800:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"8.0","cpu_average":"6","mem":"0.5"},"1453":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1920x1080 -i pipe: -f mpegts -s 1920x1080 -codec:v mpeg1video -q 1 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"1454":{"cmdline":"frigate.capture:CAM10","cpu":"9.0","cpu_average":"7","mem":"0.4"},"1456":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.13.0-d0413e3 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp:https://localhost:8554/CAM8-02 -r 10 -vf fps=10,scale_cuda=w=3200:h=1800:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"7.9","cpu_average":"6","mem":"0.5"},"1478":{"cmdline":"frigate.capture:CAM11","cpu":"8.3","cpu_average":"8","mem":"0.4"},"1480":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.13.0-d0413e3 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp:https://localhost:8554/CAM10 -r 10 -vf fps=10,scale_cuda=w=3840:h=2160:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"16.0","cpu_average":"13","mem":"0.4"},"1483":{"cmdline":"frigate.capture:CAM12","cpu":"6.9","cpu_average":"5","mem":"0.4"},"1484":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.13.0-d0413e3 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp:https://localhost:8554/CAM11 -r 10 -vf fps=10,scale_cuda=w=3840:h=2160:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"14.3","cpu_average":"13","mem":"0.4"},"1488":{"cmdline":"frigate.capture:CAM13","cpu":"8.6","cpu_average":"7","mem":"0.4"},"1490":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.13.0-d0413e3 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp:https://localhost:8554/CAM12 -r 10 -vf fps=10,scale_cuda=w=3200:h=1800:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"8.1","cpu_average":"6","mem":"0.4"},"1493":{"cmdline":"frigate.capture:CAM14","cpu":"8.2","cpu_average":"8","mem":"0.4"},"1494":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.13.0-d0413e3 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp:https://localhost:8554/CAM13 -r 10 -vf fps=10,scale_cuda=w=3840:h=2160:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"15.6","cpu_average":"12","mem":"0.5"},"1500":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.13.0-d0413e3 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp:https://localhost:8554/CAM14 -r 10 -vf fps=10,scale_cuda=w=3840:h=2160:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"14.5","cpu_average":"14","mem":"0.5"},"2897":{"cmdline":"sleep 30s","cpu":"0.0","cpu_average":"0","mem":"0.0"}},"detection_fps":99.4,"detectors":{"Coral1":{"detection_start":1690120391.039543,"inference_speed":6.0,"pid":1314},"Coral2":{"detection_start":0.0,"inference_speed":6.02,"pid":1316}},"gpu_usages":{"NVIDIA GeForce RTX 4080":{"gpu":"9.0%","mem":"54.84%"},"intel-vaapi":{"gpu":"1.68%","mem":"-%"}},"processes":{"go2rtc":{"pid":88},"logger":{"pid":1302},"recording":{"pid":1304}},"service":{"last_updated":1690120391,"latest_version":"disabled","storage":{"/dev/shm":{"free":3906.8,"mount_type":"tmpfs","total":4096.0,"used":189.2},"/media/frigate/clips":{"free":1002709.5,"mount_type":"cifs","total":1039286.4,"used":36576.9},"/media/frigate/recordings":{"free":1002709.5,"mount_type":"cifs","total":1039286.4,"used":36576.9},"/tmp/cache":{"free":953.7,"mount_type":"tmpfs","total":953.7,"used":0.0}},"temperatures":{"apex_0":44.05,"apex_1":43.05},"uptime":1024,"version":"0.13.0-d0413e3"}}

Operating system

Proxmox

Install method

Docker Compose

Coral version

M.2

Network connection

Wired

Camera make and model

Hikvision DS-2CD3367WDV3-L

Any other information that may be helpful

No response
@NickM-27
Copy link
Sponsor Collaborator

What browser are you using? Something is breaking the websocket connection between the frontend and the backend

@LckySndays
Copy link
Author

@NickM-27 lot of different users with different kind of browsers from desktop to smartphone browsers, I'm not sure how to point each of them out.

@hawkeye217
Copy link
Collaborator

I've actually noticed this on 0.13.x as well - I haven't been able to pin it down, but I haven't really tried to look deeply into it as I'm still on 0.12 on my production machine. It's happened several times as my Mac/Firefox has gone to sleep with a camera streaming. I'll return a while later to a semi-crashed Frigate with a similar stack trace.

@NickM-27
Copy link
Sponsor Collaborator

At the very least the exception should be caught, but I've not seen this at all. I use mac + chrome and have definitely had computer fall asleep while streaming a camera before

@hawkeye217
Copy link
Collaborator

At the very least the exception should be caught, but I've not seen this at all. I use mac + chrome and have definitely had computer fall asleep while streaming a camera before

Right. When it happened to me a few weeks back I looked briefly at a commit history and didn't see any changes to the websocket code since 0.12. So I figured it was something else, and didn't dig any deeper.

@hawkeye217
Copy link
Collaborator

@LckySndays Just curious, are you using Frigate on Fully Kiosk browser on a tablet?

@LckySndays
Copy link
Author

@hawkeye217 I'm pretty sure no one using kiosk on my end, so it might not be the problem.

@LckySndays

This comment was marked as outdated.

Sign in to view
@LckySndays

This comment was marked as outdated.

Sign in to view
@LckySndays
Copy link
Author

LckySndays commented Aug 1, 2023 •
edited

2023-08-01 ghcr.io/blakeblackshear/frigate:dev-7d3e5c9 (Exception in thread Thread-XX)

2023-08-01 21:15:11.485299454  Exception in thread Thread-14:
2023-08-01 21:15:11.485325741  Traceback (most recent call last):
2023-08-01 21:15:11.485330055    File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
2023-08-01 21:15:11.486185495      self.run()[2023-08-01 21:15:11] ws4py                          ERROR   : Failed to receive data
2023-08-01 21:15:11.486186381  Traceback (most recent call last):
2023-08-01 21:15:11.486187230    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 394, in once
2023-08-01 21:15:11.486187968      b = self.sock.recv(self.reading_buffer_size)
2023-08-01 21:15:11.486188599  ConnectionResetError: [Errno 104] Connection reset by peer
2023-08-01 21:15:11.486189056  
2023-08-01 21:15:11.486192404    File "/opt/frigate/frigate/output.py", line 210, in run
2023-08-01 21:15:11.486241377      ws.send(buf, binary=True)
2023-08-01 21:15:11.486244575    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 303, in send
2023-08-01 21:15:11.486293480      self._write(m)
2023-08-01 21:15:11.486296553    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 285, in _write
2023-08-01 21:15:11.486337764      self.sock.sendall(b)
2023-08-01 21:15:11.486349702  BrokenPipeError: [Errno 32] Broken pipe

2023-07-08 ghcr.io/blakeblackshear/frigate:dev-88fc0fa (Exception in thread Thread-XX)

2023-07-29 19:19:42.297372560  Exception in thread Thread-13:
2023-07-29 19:19:42.297384770  Traceback (most recent call last):
2023-07-29 19:19:42.297385386    File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
2023-07-29 19:19:42.297526713      self.run()
2023-07-29 19:19:42.297527275    File "/opt/frigate/frigate/output.py", line 210, in run
2023-07-29 19:19:42.297766194      ws.send(buf, binary=True)
2023-07-29 19:19:42.297768406    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 303, in send
2023-07-29 19:19:42.297804021      self._write(m)
2023-07-29 19:19:42.297804555    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 285, in _write
2023-07-29 19:19:42.297823791      self.sock.sendall(b)
2023-07-29 19:19:42.297839309  ConnectionResetError: [Errno 104] Connection reset by peer

2023-07-08 ghcr.io/blakeblackshear/frigate:dev-d6f82f9 (Exception in thread Thread-XX)

2023-07-29 20:19:39.116971019  Exception in thread Thread-13:
2023-07-29 20:19:39.116977058  Traceback (most recent call last):
2023-07-29 20:19:39.116977698    File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
2023-07-29 20:19:39.117226328      self.run()
2023-07-29 20:19:39.117226891    File "/opt/frigate/frigate/output.py", line 210, in run
2023-07-29 20:19:39.117260841      ws.send(buf, binary=True)
2023-07-29 20:19:39.117261401    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 303, in send
2023-07-29 20:19:39.117289944      self._write(m)
2023-07-29 20:19:39.117290443    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 285, in _write
2023-07-29 20:19:39.117309139      self.sock.sendall(b)
2023-07-29 20:19:39.117318123  ConnectionResetError: [Errno 104] Connection reset by peer

2023-07-07 ghcr.io/blakeblackshear/frigate:dev-00b9a49
(Exception in thread Thread-XX) (Confirmed by @johnvprcam24573455623 #7395)

2023-08-04 18:28:17.947936023  Exception in thread detected_frames_processor:
2023-08-04 18:28:17.948012350  Traceback (most recent call last):
2023-08-04 18:28:17.948018695    File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
2023-08-04 18:28:17.949960841      self.run()
2023-08-04 18:28:17.949971135    File "/opt/frigate/frigate/object_processing.py", line 1028, in run
2023-08-04 18:28:17.950621641      self.video_output_queue.put(
2023-08-04 18:28:17.950631499    File "/opt/frigate/frigate/util/builtin.py", line 91, in put
2023-08-04 18:28:17.950951430      raise Full
2023-08-04 18:28:17.950957499  queue.Full
2023-08-04 18:28:34.593501039  Exception in thread Thread-15:
2023-08-04 18:28:34.593557240  Traceback (most recent call last):
2023-08-04 18:28:34.593563990    File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
2023-08-04 18:28:34.594446909      self.run()
2023-08-04 18:28:34.594456447    File "/opt/frigate/frigate/output.py", line 210, in run
2023-08-04 18:28:34.594740461      ws.send(buf, binary=True)
2023-08-04 18:28:34.594747612    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 303, in send
2023-08-04 18:28:34.595873880      self._write(m)
2023-08-04 18:28:34.595883910    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 285, in _write
2023-08-04 18:28:34.596159087      self.sock.sendall(b)
2023-08-04 18:28:34.596236084  ConnectionResetError: [Errno 104] Connection reset by peer

2023-07-07 ghcr.io/blakeblackshear/frigate:dev-5259d51
(Different Error : queue.Full Fixed in #7181 on 2023-07-16) (Need Retesting for Exception in thread Thread-XX)

2023-08-02 00:57:54.135709124  Exception in thread detected_frames_processor:
2023-08-02 00:57:54.135721427  Traceback (most recent call last):
2023-08-02 00:57:54.135722476    File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
2023-08-02 00:57:54.135863578      self.run()
2023-08-02 00:57:54.135865598    File "/opt/frigate/frigate/object_processing.py", line 1028, in run
2023-08-02 00:57:54.135948392      self.video_output_queue.put(
2023-08-02 00:57:54.135949417    File "/opt/frigate/frigate/util/builtin.py", line 91, in put
2023-08-02 00:57:54.135969051      raise Full
2023-08-02 00:57:54.135979124  queue.Full

2023-07-07 ghcr.io/blakeblackshear/frigate:dev-dd02958
(Different Error : queue.Full Fixed in #7181 on 2023-07-16) (Need Retesting for Exception in thread Thread-XX)

2023-08-02 05:08:42.699287690  Exception in thread detected_frames_processor:
2023-08-02 05:08:42.699304261  Traceback (most recent call last):
2023-08-02 05:08:42.699305202    File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
2023-08-02 05:08:42.700009779      self.run()
2023-08-02 05:08:42.700014988    File "/opt/frigate/frigate/object_processing.py", line 1028, in run
2023-08-02 05:08:42.700408285      self.video_output_queue.put(
2023-08-02 05:08:42.700438431    File "/opt/frigate/frigate/util/builtin.py", line 91, in put
2023-08-02 05:08:42.700596760      raise Full
2023-08-02 05:08:42.700616533  queue.Full

2023-07-06 ghcr.io/blakeblackshear/frigate:dev-baf671b
(Different Error : Thread 0x00007f1eXXXXXXXX) (Need Retesting for Exception in thread Thread-XX)

2023-08-01 01:23:08.054407773  Fatal Python error: Bus error
2023-08-01 01:23:08.054411111  
2023-08-01 01:23:08.054411959  Current thread 0x00007f1e0cfed6c0 (most recent call first):
2023-08-01 01:23:08.054422444    File "/opt/frigate/frigate/video.py", line 192 in capture_frames
2023-08-01 01:23:08.054423154    File "/opt/frigate/frigate/video.py", line 416 in run
2023-08-01 01:23:08.054426984    File "/usr/lib/python3.9/threading.py", line 954 in _bootstrap_inner
2023-08-01 01:23:08.054435270    File "/usr/lib/python3.9/threading.py", line 912 in _bootstrap
2023-08-01 01:23:08.054437933  
2023-08-01 01:23:08.054438486  Thread 0x00007f1e0f7ee6c0 (most recent call first):
2023-08-01 01:23:08.054451000    File "/usr/lib/python3.9/multiprocessing/synchronize.py", line 261 in wait
2023-08-01 01:23:08.054465189    File "/usr/lib/python3.9/multiprocessing/synchronize.py", line 349 in wait
2023-08-01 01:23:08.054473353    File "/opt/frigate/frigate/video.py", line 268 in run
2023-08-01 01:23:08.054487152    File "/usr/lib/python3.9/threading.py", line 954 in _bootstrap_inner
2023-08-01 01:23:08.054498185    File "/usr/lib/python3.9/threading.py", line 912 in _bootstrap
2023-08-01 01:23:08.054498622  
2023-08-01 01:23:08.054499154  Thread 0x00007f1e11fef6c0 (most recent call first):
2023-08-01 01:23:08.054509973    File "/opt/frigate/frigate/log.py", line 95 in run
2023-08-01 01:23:08.054522982    File "/usr/lib/python3.9/threading.py", line 954 in _bootstrap_inner
2023-08-01 01:23:08.054533569    File "/usr/lib/python3.9/threading.py", line 912 in _bootstrap
2023-08-01 01:23:08.054533974  
2023-08-01 01:23:08.054536795  Thread 0x00007f1ea7cd3740 (most recent call first):
2023-08-01 01:23:08.054547230    File "/usr/lib/python3.9/threading.py", line 1049 in _wait_for_tstate_lock
2023-08-01 01:23:08.054555668    File "/usr/lib/python3.9/threading.py", line 1033 in join
2023-08-01 01:23:08.054566778    File "/opt/frigate/frigate/video.py", line 452 in capture_camera
2023-08-01 01:23:08.054583157    File "/usr/lib/python3.9/multiprocessing/process.py", line 108 in run
2023-08-01 01:23:08.054597320    File "/usr/lib/python3.9/multiprocessing/process.py", line 315 in _bootstrap
2023-08-01 01:23:10.056722162    File "/usr/lib/python3.9/multiprocessing/popen_fork.py", line 71 in _launch
2023-08-01 01:23:10.056724930    File "/usr/lib/python3.9/multiprocessing/popen_fork.py", line 19 in __init__
2023-08-01 01:23:10.056725807    File "/usr/lib/python3.9/multiprocessing/context.py", line 277 in _Popen
2023-08-01 01:23:10.056726449    File "/usr/lib/python3.9/multiprocessing/context.py", line 224 in _Popen
2023-08-01 01:23:10.056727076    File "/usr/lib/python3.9/multiprocessing/process.py", line 121 in start
2023-08-01 01:23:10.056727717    File "/opt/frigate/frigate/app.py", line 430 in start_camera_capture_processes
2023-08-01 01:23:10.056728319    File "/opt/frigate/frigate/app.py", line 540 in start
2023-08-01 01:23:10.056728895    File "/opt/frigate/frigate/__main__.py", line 17 in <module>
2023-08-01 01:23:10.056729452    File "/usr/lib/python3.9/runpy.py", line 87 in _run_code
2023-08-01 01:23:10.056776779    File "/usr/lib/python3.9/runpy.py", line 197 in _run_module_as_main

2023-07-06 ghcr.io/blakeblackshear/frigate:dev-2fae9dc (Cannot start properly)
2023-07-03 ghcr.io/blakeblackshear/frigate:dev-12d4a47 (Exception in thread Thread-XX)

2023-08-05 21:08:39.973360292  Exception in thread Thread-13:
2023-08-05 21:08:39.973369905  Traceback (most recent call last):
2023-08-05 21:08:39.973370749    File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
2023-08-05 21:08:39.974115564      self.run()
2023-08-05 21:08:39.974119587    File "/opt/frigate/frigate/output.py", line 151, in run
2023-08-05 21:08:39.974162445      ws.send(buf, binary=True)
2023-08-05 21:08:39.974163208    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 303, in send
2023-08-05 21:08:39.974413546      self._write(m)
2023-08-05 21:08:39.974417200    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 285, in _write
2023-08-05 21:08:39.974766886      self.sock.sendall(b)
2023-08-05 21:08:39.974820826  BrokenPipeError: [Errno 32] Broken pipe
2023-08-05 21:08:39.975056753  [2023-08-05 21:08:39] ws4py                          ERROR   : Failed to receive data
2023-08-05 21:08:39.975057925  Traceback (most recent call last):
2023-08-05 21:08:39.975058625    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 394, in once
2023-08-05 21:08:39.975059225      b = self.sock.recv(self.reading_buffer_size)
2023-08-05 21:08:39.975059849  ConnectionResetError: [Errno 104] Connection reset by peer

2023-07-02 ghcr.io/blakeblackshear/frigate:dev-83edf95 (Exception in thread Thread-XX)

2023-08-08 21:57:22.625239976  Exception in thread Thread-13:
2023-08-08 21:57:22.625246635  Traceback (most recent call last):
2023-08-08 21:57:22.625253396    File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
2023-08-08 21:57:22.626207214      self.run()
2023-08-08 21:57:22.626211538    File "/opt/frigate/frigate/output.py", line 151, in run
2023-08-08 21:57:22.626416799  [2023-08-08 21:57:22] ws4py                          ERROR   : Failed to receive data
2023-08-08 21:57:22.626417415  Traceback (most recent call last):
2023-08-08 21:57:22.626418112    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 394, in once
2023-08-08 21:57:22.626418698      b = self.sock.recv(self.reading_buffer_size)
2023-08-08 21:57:22.626419312  ConnectionResetError: [Errno 104] Connection reset by peer
2023-08-08 21:57:22.626422208      ws.send(buf, binary=True)
2023-08-08 21:57:22.626424862    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 303, in send
2023-08-08 21:57:22.626474767      self._write(m)
2023-08-08 21:57:22.626477800    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 285, in _write
2023-08-08 21:57:22.626518310      self.sock.sendall(b)
2023-08-08 21:57:22.626530486  BrokenPipeError: [Errno 32] Broken pipe

2023-06-30 ghcr.io/blakeblackshear/frigate:dev-f3f9b36 (Exception in thread Thread-XX)

2023-08-09 20:08:04.214526443  Exception in thread Thread-12:
2023-08-09 20:08:04.214597455  Traceback (most recent call last):
2023-08-09 20:08:04.214601733    File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
2023-08-09 20:08:04.216295836      self.run()
2023-08-09 20:08:04.216333314    File "/opt/frigate/frigate/output.py", line 151, in run
2023-08-09 20:08:04.216571528      ws.send(buf, binary=True)
2023-08-09 20:08:04.216575026    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 303, in send
2023-08-09 20:08:04.216612688  [2023-08-09 20:08:04] ws4py                          ERROR   : Failed to receive data
2023-08-09 20:08:04.216613452  Traceback (most recent call last):
2023-08-09 20:08:04.216614204    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 394, in once
2023-08-09 20:08:04.216614851      b = self.sock.recv(self.reading_buffer_size)
2023-08-09 20:08:04.216615502  ConnectionResetError: [Errno 104] Connection reset by peer
2023-08-09 20:08:04.216633304      self._write(m)
2023-08-09 20:08:04.216636546    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 285, in _write
2023-08-09 20:08:04.216680290      self.sock.sendall(b)
2023-08-09 20:08:04.216755498  BrokenPipeError: [Errno 32] Broken pipe

2023-06-19 ghcr.io/blakeblackshear/frigate:dev-7c1568f (Exception in thread Thread-XX)

2023-08-08 00:06:59.511801224  Exception in thread Thread-23:
2023-08-08 00:06:59.511824190  Traceback (most recent call last):
2023-08-08 00:06:59.511825291    File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
2023-08-08 00:06:59.512682625      self.run()
2023-08-08 00:06:59.512684575    File "/opt/frigate/frigate/output.py", line 151, in run
2023-08-08 00:06:59.512965646      ws.send(buf, binary=True)
2023-08-08 00:06:59.512971289    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 303, in send
2023-08-08 00:06:59.513312595      self._write(m)
2023-08-08 00:06:59.513314427    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 285, in _write
2023-08-08 00:06:59.513370007      self.sock.sendall(b)
2023-08-08 00:06:59.513556230  ConnectionResetError: [Errno 104] Connection reset by peer

@LckySndays LckySndays mentioned this issue Aug 5, 2023
Closed
@LckySndays

This comment was marked as outdated.

Sign in to view
@blakeblackshear
Copy link
Owner

These "Connection reset by peer" messages happen during normal usage of the frontend when the websocket is disconnected. It is not the source of the problems you are seeing and can be ignored.

@blakeblackshear
Copy link
Owner

The brokenpipe error looks like the useful log message.

@LckySndays
Copy link
Author

The bad news is the brokenpipe error is still showing up even on recent build dev-7d3e5c9 (2023-08-01)

And yes the "Connection reset by peer" is not the source of problem but the "Exception in thread Thread-XX" follow up into
"Connection reset by peer" is practically mean the same thing by my testing

Since sometimes the error log is just stop there without having the chance to log the "BrokenPipeError: [Errno 32] Broken pipe" afterward, of course it still need more testing to make sure all of this

@NickM-27 NickM-27 added bug Something isn't working and removed support triage labels Aug 18, 2023
@NickM-27 NickM-27 mentioned this issue Aug 18, 2023
Closed
@spacebares
Copy link
Contributor

Showing up still in dev-3921a7f

2023-08-18 16:54:33.364017969  Exception in thread Thread-16:
2023-08-18 16:54:33.364031065  Traceback (most recent call last):
2023-08-18 16:54:33.364037254    File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
2023-08-18 16:54:33.392749636      self.run()
2023-08-18 16:54:33.392760740    File "/opt/frigate/frigate/output.py", line 210, in run
2023-08-18 16:54:33.393212817      ws.send(buf, binary=True)
2023-08-18 16:54:33.393221709    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 303, in send
2023-08-18 16:54:33.393551228      self._write(m)
2023-08-18 16:54:33.393558687    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 285, in _write
2023-08-18 16:54:33.393858173      self.sock.sendall(b)
2023-08-18 16:54:33.394400050  BrokenPipeError: [Errno 32] Broken pipe
2023-08-18 16:54:33.436108830  [2023-08-18 16:54:33] ws4py                          ERROR   : Failed to receive data
2023-08-18 16:54:33.436119428  Traceback (most recent call last):
2023-08-18 16:54:33.436126513    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 394, in once
2023-08-18 16:54:33.436131966      b = self.sock.recv(self.reading_buffer_size)
2023-08-18 16:54:33.436137324  ConnectionResetError: [Errno 104] Connection reset by peer

@NickM-27
Copy link
Sponsor Collaborator

It seems clear where the error is coming from, just not sure if it can be caught without consequences or more investigation needs to happen. I have been entirely unable to reproduce this issue myself. One thing I notice is that i have birdseye restream enabled, so perhaps there is some issue when it is not enabled.

@LckySndays
Copy link
Author

I have been running the stable again for a full week with same config etc, to make sure its not happening there and the result is that it run greatly without crashing at all.

Below is the the earliest recorded Exception in thread Thread-XX logs from my testing.

2023-06-30 ghcr.io/blakeblackshear/frigate:dev-f3f9b36 (Exception in thread Thread-XX with BrokenPipe)

2023-08-09 16:16:23.000766873  [2023-08-09 16:16:23] frigate.object_processing      WARNING : Unable to create jpg because frame 1691597782.881278 is not in the cache
2023-08-09 20:08:04.214526443  Exception in thread Thread-12:
2023-08-09 20:08:04.214597455  Traceback (most recent call last):
2023-08-09 20:08:04.214601733    File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
2023-08-09 20:08:04.216295836      self.run()
2023-08-09 20:08:04.216333314    File "/opt/frigate/frigate/output.py", line 151, in run
2023-08-09 20:08:04.216571528      ws.send(buf, binary=True)
2023-08-09 20:08:04.216575026    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 303, in send
2023-08-09 20:08:04.216612688  [2023-08-09 20:08:04] ws4py                          ERROR   : Failed to receive data
2023-08-09 20:08:04.216613452  Traceback (most recent call last):
2023-08-09 20:08:04.216614204    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 394, in once
2023-08-09 20:08:04.216614851      b = self.sock.recv(self.reading_buffer_size)
2023-08-09 20:08:04.216615502  ConnectionResetError: [Errno 104] Connection reset by peer
2023-08-09 20:08:04.216633304      self._write(m)
2023-08-09 20:08:04.216636546    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 285, in _write
2023-08-09 20:08:04.216680290      self.sock.sendall(b)
2023-08-09 20:08:04.216755498  BrokenPipeError: [Errno 32] Broken pipe

2023-06-14 ghcr.io/blakeblackshear/frigate:dev-ff1b5c8 (Exception in thread Thread-XX without BrokenPipe)

2023-08-11 03:46:49.985547221  [2023-08-11 03:46:49] frigate.object_processing      WARNING : Unable to create jpg because frame 1691725609.476395 is not in the cache
2023-08-11 03:52:07.050882244  Exception in thread Thread-13:
2023-08-11 03:52:07.050893791  Traceback (most recent call last):
2023-08-11 03:52:07.050898153    File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
2023-08-11 03:52:07.051611563      self.run()
2023-08-11 03:52:07.051613134    File "/opt/frigate/frigate/output.py", line 150, in run
2023-08-11 03:52:07.051751114      ws.send(buf, binary=True)
2023-08-11 03:52:07.051752434    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 303, in send
2023-08-11 03:52:07.052008773      self._write(m)
2023-08-11 03:52:07.052011191    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 285, in _write
2023-08-11 03:52:07.052048163      self.sock.sendall(b)
2023-08-11 03:52:07.052188564  ConnectionResetError: [Errno 104] Connection reset by peer

@NickM-27
Copy link
Sponsor Collaborator

NickM-27 commented Aug 19, 2023 •
edited

Difficulty is that both of those commits are unrelated to output.py and websockets in general so it would seem unlikely that they directly caused it.

In any case, I think the more important part is why some users are seeing it and others are not, I have not seen this even one time.

I'm going to create a PR to catch this and print debug info, perhaps it won't have any consequences catching and ignoring and at least won't caught a failure

#7525

@LckySndays
Copy link
Author

LckySndays commented Aug 20, 2023 •
edited

2023-06-12 ghcr.io/blakeblackshear/frigate:dev-ff90db3 (Exception in thread Thread-XX with BrokenPipe)

2023-08-19 20:39:57.585608547  Exception in thread Thread-13:
2023-08-19 20:39:57.585618594  Traceback (most recent call last):
2023-08-19 20:39:57.585619633    File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
2023-08-19 20:39:57.586280316      self.run()
2023-08-19 20:39:57.586283766    File "/opt/frigate/frigate/output.py", line 150, in run
2023-08-19 20:39:57.586453251  [2023-08-19 20:39:57] ws4py                          ERROR   : Failed to receive data
2023-08-19 20:39:57.586454075  Traceback (most recent call last):
2023-08-19 20:39:57.586454909    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 394, in once
2023-08-19 20:39:57.586455578      b = self.sock.recv(self.reading_buffer_size)
2023-08-19 20:39:57.586456234  ConnectionResetError: [Errno 104] Connection reset by peer
2023-08-19 20:39:57.586526886      ws.send(buf, binary=True)
2023-08-19 20:39:57.586533287    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 303, in send
2023-08-19 20:39:57.586791691      self._write(m)
2023-08-19 20:39:57.586793284    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 285, in _write
2023-08-19 20:39:57.586794057      self.sock.sendall(b)
2023-08-19 20:39:57.586794679  BrokenPipeError: [Errno 32] Broken pipe

I only have 3 more build to test out
2023-06-12 ghcr.io/blakeblackshear/frigate:dev-ff90db3 (Exception in thread Thread-XX with BrokenPipe)
2023-06-12 ghcr.io/blakeblackshear/frigate:master-bd9942bs (Skipped for now different branch)
2023-06-12 ghcr.io/blakeblackshear/frigate:dev-dfd574b (Test in progress)
2023-06-11 ghcr.io/blakeblackshear/frigate:dev-0f1cf67 (Waiting to test - output.py related)
2023-06-11 ghcr.io/blakeblackshear/frigate:0.12.1 (OK)

@spacebares
Copy link
Contributor

spacebares commented Aug 21, 2023 •
edited

It seems clear where the error is coming from, just not sure if it can be caught without consequences or more investigation needs to happen. I have been entirely unable to reproduce this issue myself. One thing I notice is that i have birdseye restream enabled, so perhaps there is some issue when it is not enabled.

I have enabled birdseye restream 3 days ago and the issue has gone away. Im still on dev-3921a7f which has the issue with birdseye restream disabled.

Edit: spoke too soon,

2023-08-22 03:20:36.205607578  Exception in thread Thread-16:
2023-08-22 03:20:36.205984756  Traceback (most recent call last):
2023-08-22 03:20:36.207255386    File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
2023-08-22 03:20:36.209537677      self.run()
2023-08-22 03:20:36.209684758    File "/opt/frigate/frigate/output.py", line 210, in run
2023-08-22 03:20:36.210056959      ws.send(buf, binary=True)
2023-08-22 03:20:36.210061191    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 303, in send
2023-08-22 03:20:36.210766912      self._write(m)
2023-08-22 03:20:36.210787675    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 285, in _write
2023-08-22 03:20:36.210975479      self.sock.sendall(b)
2023-08-22 03:20:36.211851267  ConnectionResetError: [Errno 104] Connection reset by peer

for the record, my frigate box has a docker restart frigate every 24hours.

@spacebares
Copy link
Contributor

I'm going to create a PR to catch this and print debug info, perhaps it won't have any consequences catching and ignoring and at least won't caught a failure

#7525

received same broken camera behavior, but different error. Instead of connection reset i get broken pipe.
This is with dev-767abed

2023-08-22 14:39:04.221014656  Exception in thread Thread-16:
2023-08-22 14:39:04.221025521  Traceback (most recent call last):
2023-08-22 14:39:04.221031475    File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
2023-08-22 14:39:04.225061120      self.run()
2023-08-22 14:39:04.225120159    File "/opt/frigate/frigate/output.py", line 210, in run
2023-08-22 14:39:04.233326024      ws.send(buf, binary=True)
2023-08-22 14:39:04.233334283    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 303, in send
2023-08-22 14:39:04.234043334      self._write(m)
2023-08-22 14:39:04.234271306    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 285, in _write
2023-08-22 14:39:04.234864928      self.sock.sendall(b)
2023-08-22 14:39:04.235549993  BrokenPipeError: [Errno 32] Broken pipe

@NickM-27
Copy link
Sponsor Collaborator

Put up another PR to catch both

@LckySndays
Copy link
Author

I hope that we can just catch and ignore that without consequences, but in case that if we need to deal with it, I'm pretty sure that the cause of issue is this commit dev-ff90db3

So far I've been testing 1 previous build before that commit for 3 days and it still running just fine

@TulkasLord
Copy link

TulkasLord commented Feb 5, 2024 •
edited

Hi guys,
I just got the same error over here after upgrading from version 0.12.1 to the latest(as Today) version 0.13.1.

Only affects the Birdseye.

image
(individual cameras and other features seem ok.)

Frigate.yml "Birdseye" section:

...
birdseye:
  enabled: true
  width: 3072
  height: 2048
  quality: 4
  mode: continuous
  restream: false  
...

Frigate Logs only show this exception several times.

2024-02-04 23:22:05.541874931  [INFO] Preparing Frigate...
2024-02-04 23:22:05.810167389  [INFO] Starting Frigate...
2024-02-04 23:22:11.120082110  [2024-02-04 23:22:11] frigate.app                    INFO    : Starting Frigate (0.13.1-34fb1c2)
2024-02-04 23:22:11.120090647  [2024-02-04 23:22:11] frigate.app                    INFO    : Creating directory: /config/model_cache
2024-02-04 23:22:11.392611344  [2024-02-04 23:22:11] peewee_migrate.logs            INFO    : Starting migrations
...
...
...
2024-02-04 23:24:54.164849393  Exception in thread Thread-26:
2024-02-04 23:24:54.164857072  Traceback (most recent call last):
2024-02-04 23:24:54.164859304    File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
2024-02-04 23:24:54.195845781      self.run()
2024-02-04 23:24:54.195853502    File "/opt/frigate/frigate/output.py", line 255, in run
2024-02-04 23:24:54.196076154      ws.send(buf, binary=True)
2024-02-04 23:24:54.196079631    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 303, in send
2024-02-04 23:24:54.197289781      self._write(m)
2024-02-04 23:24:54.197294757    File "/usr/local/lib/python3.9/dist-packages/ws4py/websocket.py", line 285, in _write
2024-02-04 23:24:54.197296340      self.sock.sendall(b)
2024-02-04 23:24:54.197297988  OSError: [Errno 9] Bad file descriptor
...

The running instance of Frigate uses docker-compose. No errors there.

● frigate.service - Docker Compose Application Service
     Loaded: loaded (/etc/systemd/system/frigate.service; enabled; preset: disabled)
     Active: active (exited) since Sun 2024-02-04 23:22:05 EST; 51min ago
    Process: 1651 ExecStart=/usr/local/bin/docker-compose up -d (code=exited, status=0/SUCCESS)
   Main PID: 1651 (code=exited, status=0/SUCCESS)
        CPU: 116ms

docker-compose logs -f frigate | more

frigate  | s6-rc: info: service s6rc-fdholder: starting
frigate  | s6-rc: info: service s6rc-oneshot-runner: starting
frigate  | s6-rc: info: service s6rc-oneshot-runner successfully started
frigate  | s6-rc: info: service fix-attrs: starting
frigate  | s6-rc: info: service s6rc-fdholder successfully started
frigate  | s6-rc: info: service fix-attrs successfully started
frigate  | s6-rc: info: service legacy-cont-init: starting
frigate  | s6-rc: info: service legacy-cont-init successfully started
frigate  | s6-rc: info: service log-prepare: starting
frigate  | s6-rc: info: service log-prepare successfully started
frigate  | s6-rc: info: service nginx-log: starting
frigate  | s6-rc: info: service go2rtc-log: starting
frigate  | s6-rc: info: service frigate-log: starting
frigate  | s6-rc: info: service nginx-log successfully started
frigate  | s6-rc: info: service frigate-log successfully started
frigate  | s6-rc: info: service go2rtc-log successfully started
frigate  | s6-rc: info: service go2rtc: starting
frigate  | s6-rc: info: service go2rtc successfully started
frigate  | s6-rc: info: service go2rtc-healthcheck: starting
frigate  | s6-rc: info: service frigate: starting
frigate  | s6-rc: info: service frigate successfully started
frigate  | s6-rc: info: service nginx: starting
frigate  | s6-rc: info: service go2rtc-healthcheck successfully started
frigate  | s6-rc: info: service nginx successfully started
frigate  | s6-rc: info: service legacy-services: starting
frigate  | 2024-02-04 23:22:05.541874931  [INFO] Preparing Frigate...
frigate  | 2024-02-04 23:22:05.548417993  [INFO] Starting NGINX...
frigate  | s6-rc: info: service legacy-services successfully started
frigate  | 2024-02-04 23:22:05.810167389  [INFO] Starting Frigate...
frigate  | 2024-02-04 23:22:05.810477330  [INFO] Removing stale config from last run...
frigate  | rm: cannot remove '/dev/shm/go2rtc.yaml': Device or resource busy
frigate  | 2024-02-04 23:22:05.821179991  [INFO] The go2rtc service exited with code 1 (by signal 0)
frigate  | 2024-02-04 23:22:06.853657600  [INFO] Removing stale config from last run...
frigate  | rm: cannot remove '/dev/shm/go2rtc.yaml': Device or resource busy
frigate  | 2024-02-04 23:22:06.865831658  [INFO] The go2rtc service exited with code 1 (by signal 0)
frigate  | 2024-02-04 23:22:07.885150042  [INFO] Removing stale config from last run...
frigate  | rm: cannot remove '/dev/shm/go2rtc.yaml': Device or resource busy
...

There is anything that I am missing in the release notes about any Birdseye changes in the config file or anywhere else?
🤔

txs in advance...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working pinned
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@blakeblackshear @TulkasLord @LckySndays @NickM-27 @hawkeye217 @spacebares