[HW Accel Support]: AttributeError: 'TensorRtDetector' object has no attribute 'outputs'

[sbandersen14]

Describe the problem you are having

After rebooting my ProxMox server the feeds into frigate stopped working. I tried to rebuild the tensorrt file and it didn't help.

Version

0.13.2-6476F8A

Frigate config file

mqtt:
  host: 10.100.1.14
  port: 1883
  user: ***
  password: ***

cameras:
  FrontDoor: # <------ Name the camera
    enabled: true
    ffmpeg:
      hwaccel_args: preset-nvidia-h264
      inputs:
      - path: rtsp:https://10.100.1.50:554/detect   # <----- The stream you want to use for detection
        roles:
        - detect
      - path: rtsp:https://10.100.1.50:554/main   # <----- The stream you want to use for recording
        roles:
        - audio
        - record
    detect:
      width: 640
      height: 480
    record:
      enabled: true
      retain:
        days: 3
        mode: all
      events:
        required_zones:
        - frontSteps
        retain:
          default: 30
          mode: motion
    snapshots:
      enabled: true
      retain:
        default: 30
    motion:
      mask:
      - 640,0,640,146,0,75,0,0
    zones:
      frontSteps:
        coordinates: 0,480,490,480,481,271,251,212,0,235
    objects:
      filters:
        person:
          mask:
          - 640,159,640,157
  Garage: # <------ Name the camera
    enabled: true
    ffmpeg:
      hwaccel_args: preset-nvidia-h264
      inputs:
      - path: rtsp:https://10.100.1.51:554/detect   # <----- The stream you want to use for detection
        roles:
        - detect
      - path: rtsp:https://10.100.1.51:554/main   # <----- The stream you want to use for recording
        roles:
        - audio
        - record
    detect:
      width: 640
      height: 480
    record:
      enabled: true
      retain:
        days: 3
        mode: all
      events:
        retain:
          default: 30
          mode: motion
    snapshots:
      enabled: true
      retain:
        default: 30
    motion:
      mask:
      - 116,183,124,297,190,296,219,258,219,190,298,177,379,155,429,60,388,0,183,0
    objects:
      filters:
        person:
          mask:
          - 440,0,454,480
  BackDoors: # <------ Name the camera
    enabled: true
    ffmpeg:
      hwaccel_args: preset-nvidia-h264
      inputs:
      - path: rtsp:https://10.100.1.52:554/detect   # <----- The stream you want to use for detection
        roles:
        - detect
      - path: rtsp:https://10.100.1.52:554/main   # <----- The stream you want to use for recording
        roles:
        - audio
        - record
    detect:
      width: 640
      height: 480
    record:
      enabled: true
      retain:
        days: 3
        mode: all
      events:
        retain:
          default: 30
          mode: motion
    snapshots:
      enabled: true
      retain:
        default: 30

detectors:
  tensorrt:
    type: tensorrt
    device: 0 #This is the default, select the first GPU  
model:
  path: /config/model_cache/tensorrt/yolov7-320.trt
  input_tensor: nchw
  input_pixel_format: rgb
  width: 320
  height: 320

docker-compose file or Docker CLI command

version: "3.9"
services:
  frigate:
    container_name: frigate
    privileged: true # this may not be necessary for all setups
    restart: unless-stopped
    image: ghcr.io/blakeblackshear/frigate:stable-tensorrt
    deploy:    # <------------- Add this section
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1 # number of GPUs
              capabilities: [gpu]
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ./config:/config
      - /mnt/media/cctv_clips:/media/frigate
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "5000:5000"
      - "8554:8554" # RTSP feeds
      - "8555:8555/tcp" # WebRTC over tcp
      - "8555:8555/udp" # WebRTC over udp
  #environment:
  #  - YOLO_MODELS=yolov7-320

Relevant log output

2024-06-13 09:08:51.860887496  [INFO] Preparing Frigate...
2024-06-13 09:08:51.891170071  [INFO] Starting Frigate...
2024-06-13 09:08:53.819586761  [2024-06-13 09:08:53] frigate.app                    INFO    : Starting Frigate (0.13.2-6476f8a)
2024-06-13 09:08:54.179622413  [2024-06-13 09:08:54] peewee_migrate.logs            INFO    : Starting migrations
2024-06-13 09:08:54.186217058  [2024-06-13 09:08:54] peewee_migrate.logs            INFO    : There is nothing to migrate
2024-06-13 09:08:54.197986243  [2024-06-13 09:08:54] frigate.app                    INFO    : Recording process started: 299
2024-06-13 09:08:54.201800084  [2024-06-13 09:08:54] frigate.app                    INFO    : go2rtc process pid: 102
2024-06-13 09:08:54.215009906  [2024-06-13 09:08:54] frigate.record.maintainer      WARNING : Unable to keep up with recording segments in cache for FrontDoor. Keeping the 6 most recent segments out of 8 and discarding the rest...
2024-06-13 09:08:54.222786455  [2024-06-13 09:08:54] frigate.record.maintainer      WARNING : Unable to keep up with recording segments in cache for Garage. Keeping the 6 most recent segments out of 8 and discarding the rest...
2024-06-13 09:08:54.222793541  [2024-06-13 09:08:54] frigate.record.maintainer      WARNING : Unable to keep up with recording segments in cache for BackDoors. Keeping the 6 most recent segments out of 8 and discarding the rest...
2024-06-13 09:08:54.257801679  [2024-06-13 09:08:54] detector.tensorrt              INFO    : Starting detection process: 309
2024-06-13 09:08:54.268434015  [2024-06-13 09:08:54] frigate.app                    INFO    : Output process started: 311
2024-06-13 09:08:54.298609446  Process detector:tensorrt:
2024-06-13 09:08:54.299484992  Traceback (most recent call last):
2024-06-13 09:08:54.299886886    File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
2024-06-13 09:08:54.299890189      self.run()
2024-06-13 09:08:54.299891743    File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
2024-06-13 09:08:54.299893102      self._target(*self._args, **self._kwargs)
2024-06-13 09:08:54.299894364    File "/opt/frigate/frigate/object_detection.py", line 102, in run_detector
2024-06-13 09:08:54.299895674      object_detector = LocalObjectDetector(detector_config=detector_config)
2024-06-13 09:08:54.299898310    File "/opt/frigate/frigate/object_detection.py", line 53, in __init__
2024-06-13 09:08:54.299925916      self.detect_api = create_detector(detector_config)
2024-06-13 09:08:54.299927555    File "/opt/frigate/frigate/detectors/__init__.py", line 18, in create_detector
2024-06-13 09:08:54.299928798      return api(detector_config)
2024-06-13 09:08:54.299930160    File "/opt/frigate/frigate/detectors/plugins/tensorrt.py", line 204, in __init__
2024-06-13 09:08:54.299962701      assert (
2024-06-13 09:08:54.299964298  AssertionError: Failed to initialize cuda 999
2024-06-13 09:08:54.299965556  Exception ignored in: <function TensorRtDetector.__del__ at 0x73208e696d30>
2024-06-13 09:08:54.299982441  Traceback (most recent call last):
2024-06-13 09:08:54.299983980    File "/opt/frigate/frigate/detectors/plugins/tensorrt.py", line 239, in __del__
2024-06-13 09:08:54.299985274      if self.outputs is not None:
2024-06-13 09:08:54.299986520  AttributeError: 'TensorRtDetector' object has no attribute 'outputs'
2024-06-13 09:08:54.401056833  [2024-06-13 09:08:54] frigate.app                    INFO    : Camera processor started for FrontDoor: 323
2024-06-13 09:08:54.485396938  [2024-06-13 09:08:54] frigate.app                    INFO    : Camera processor started for Garage: 327
2024-06-13 09:08:54.485403437  [2024-06-13 09:08:54] frigate.app                    INFO    : Camera processor started for BackDoors: 330
2024-06-13 09:08:54.509870479  [2024-06-13 09:08:54] frigate.app                    INFO    : Capture process started for FrontDoor: 343
2024-06-13 09:08:54.509877514  [2024-06-13 09:08:54] frigate.app                    INFO    : Capture process started for Garage: 346
2024-06-13 09:08:54.533499594  [2024-06-13 09:08:54] frigate.app                    INFO    : Capture process started for BackDoors: 350
2024-06-13 09:08:59.231812286  [2024-06-13 09:08:59] frigate.video                  ERROR   : FrontDoor: Unable to read frames from ffmpeg process.
2024-06-13 09:08:59.232586824  [2024-06-13 09:08:59] frigate.video                  ERROR   : FrontDoor: ffmpeg process is not running. exiting capture thread...
2024-06-13 09:08:59.306224531  [2024-06-13 09:08:59] frigate.video                  ERROR   : BackDoors: Unable to read frames from ffmpeg process.
2024-06-13 09:08:59.306354909  [2024-06-13 09:08:59] frigate.video                  ERROR   : BackDoors: ffmpeg process is not running. exiting capture thread...
2024-06-13 09:08:59.404925505  [2024-06-13 09:08:59] frigate.video                  ERROR   : Garage: Unable to read frames from ffmpeg process.
2024-06-13 09:08:59.405183200  [2024-06-13 09:08:59] frigate.video                  ERROR   : Garage: ffmpeg process is not running. exiting capture thread...

FFprobe output from your camera

Input #0, rtsp, from 'rtsp:https://10.100.1.52:554/detect':
  Metadata:
    title           : RTSP Server(H264) - BackDoors
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: h264 (Main), yuv420p(tv, bt709, progressive), 640x480 [SAR 1:1 DAR 4:3], 5 fps, 5 tbr, 90k tbn
  Stream #0:1: Audio: pcm_mulaw, 8000 Hz, mono, s16, 64 kb/s

Operating system

Proxmox

Install method

Docker Compose

Network connection

Wired

Camera make and model

iCamera2

Any other information that may be helpful

No response

[NickM-27]

this definitely points to an issue with the driver or nvidia toolkit, perhaps something updated or got corrupted

[sbandersen14]

Any thoughts on what to do to fix it?