Frigate in Proxmox LXC - Unprivileged with Intel iGPU (11th gen), USB Coral and Network share

[Dougiebabe]

So this has taken a little while for me to get working and I'm not sure that it's 100% correct so I'm open to feedback on this

The reason I have tried to do this is because I want to use the iGPU for both Frigate and Plex on this newer 11th gen CPU. First I tried to do this in a VM with the iGPU passed through and while this seemed to work OK for Frigate (although I did have some random crashes and /TMP filling up) as soon as I started transcoding on the iGPU it would freeze the GPU and I would have to restart the VM to get things working again.
From reading this could be an issue with the newer 11th+ gen iGPUs not passing through properly? I'm not sure what the reason was but I couldn't get Plex transcoding properly on this GPU.

I started reading up on LXC containers and found that you can share resources across containers and it's talking direct to the GPU without a virtual abstraction layer, great!

First I got Frigate working with Privileged containers and then Plex, all was good. Frigate was working with HW Accel and Plex was transcoding using HW Accel as well. At this point I did some reading of Privileged vs Unprivileged containers and saw that it's recommended where possible to use Unprivileged containers and with Plex being open to the Internet I was keen to do so.

So that is the WHY I spent the last week or so trying to get this working, here is the HOW

!!! I give absolutely no warranty/deep support on what I write below. This write up is a mix of all kinds of information from all over the internet !!!

I'll split this up into a few sections:

1. Setting up a Proxmox LXC Unprivileged container
2. Passing through the USB Coral
3. Passing through the iGPU
4. Passing through the network share
5. Installing Frigate

1) Setting up a Proxmox LXC Unpriv Container

Logon to Proxmox host --> go to 'Local' on the L/H Pane --> CT Templates --> Templates

I prefer to use Ubuntu so in this guide I will be using Ubuntu 22.04, I have tested this on Debian 11 Turnkey Core and it worked so others should work

Click Templates --> search 'Ubuntu' --> download 22.04

Create a new CT (LXC Container) by right clicking on your host

Give the CT a name
Leave Unpriv and Nesting ticked
Type a strong password (The password you use here is what you will use to logon/SSH to the CT)

Choose the template

Give the CT a disk (this is used for the OS and any clips/recordings if you are NOT using a network share)

Assign the number of CPU and RAM you wish to use
Assign the network (I always forget to change the network to DHCP)

Choose DNS settings
Confirm the CT

You now have a CT ready for us to configure

2) Passing through the USB Coral

If you don't have a USB coral you can skip this part. If you have an M.2/PCIe coral then these instructions will probably not work for you.

Console onto your Proxmox host

Type lsusb and you should see something like this - Note down the USB Bus that the coral is present on

Type the following into the Proxmox console

cd /etc/pve/lxc
ls

You should now be in the lxc folder and see a list of CT conf files. We need to edit the CT conf file that corresponds to the CT we made above, you can find this in Proxmox

So I need to then type nano 114.conf to edit the conf file for the CT I am working on - YOURS MAY DIFFER

We need to add the below to the conf file - REPLACE THE BUS NUMBER ON THE LAST LINE WITH YOUR OWN

usb0: host=1a6e:089a,usb3=1
usb1: host=18d1:9302,usb3=1
lxc.cgroup2.devices.allow: c 189:* rwm
lxc.mount.entry: /dev/bus/usb/004 dev/bus/usb/004 none bind,optional,create=dir

My understanding of the above commands :

usb0: host=1a6e:089a,usb3=1
usb1: host=18d1:9302,usb3=1 - These are the devices IDs (I believe) that the coral can have and I think lock the device to the LXC

lxc.cgroup2.devices.allow: c 189:* rwm - This is allow the USB Device ID to passed through to LXC

lxc.mount.entry: /dev/bus/usb/004 dev/bus/usb/004 none bind,optional,create=dir - This mounts the device to the LXC at the path dev/bus/usb/004

Ctrl-x to save the file

That's it, if you don't need iGPU or Network shares you can move on to the Installing Frigate step!

3) Passing through the iGPU

This is the bit that got me for days and I'm still not sure that the way I am suggesting here is 100% correct.

I have tried doing idmaps and mapping the video and render group to the groups in the LXC but it just would NOT work, then I found a post on reddit while looking into using udev (which worked in a priv but not unpriv CT) which fixed the problem.

So I suggest you read the below and understand what is happening before following the steps, hopefully someone can tell me if the chmod command below is OK? I think it allows everyone to use the render128 device which should be a lower risk than using a priv CT?

The guide:

Console onto your Proxmox host

type lspci -nnv | grep VGA and confirm your GPU is detected

Type ls -l /dev/dri and you should get something like the below

Here you can see the the device 'card0' has a major number of 226 and a minor number of 0
The renderD128 has a major number of 226 and a minor number of 128

We need to tell the container that it is allowed to use these devices.

Type the following into the Proxmox console

cd /etc/pve/lxc
ls

You should now be in the lxc folder and see a list of CT conf files. We need to edit the CT conf file that corresponds to the CT we made above, you can find this in Proxmox

So I need to then type nano 114.conf to edit the conf file for the CT I am working on - YOURS MAY DIFFER

We need to add the below to the conf file - YOURS COULD BE DIFFERENT TO THIS DEPENDING ON THE ls -l /dev/dri OUTPUT
lxc.cgroup2.devices.allow: c 226:0 rwm
lxc.cgroup2.devices.allow: c 226:128 rwm
lxc.mount.entry: /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,create=file 0, 0
lxc.mount.entry: /dev/dri dev/dri none bind,optional,create=dir

My understanding of the above:

lxc.cgroup2.devices.allow: c 226:0 rwm
lxc.cgroup2.devices.allow: c 226:128 rwm - These two are the devices we want the LXC to be allowed to use

lxc.mount.entry: /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,create=file 0, 0
lxc.mount.entry: /dev/dri dev/dri none bind,optional,create=dir - These two are the mounting points within the LXC container that the devices will be mounted to

Ctrl-x to save the file

Now, at this point the LXC should have the device be mounted but it won't have the correct permissions. There are a lot of guides that will then say you need to an lxc.idmap: to your conf file. I tried and tried and tried to get this working in that way but failed everytime. The device would show the correct group (render) and vainfo would show the device but nothing within the CT could use it. Yes I made sure the maps were correct and Yes I added the required lines to the /etc/subgid file. It just didn't work. I would LOVE to get this working and if someone has a suggestion please let me know.

Anyway, this guide will be using the following command to get access to the GPU within the CT

On the Proxmox host console type chmod 666 /dev/dri/renderD128

This will give rw access to the renderD128 device for ALL users

Now I haven't tested this next part as my host hasn't been rebooted yet but the reddit post above explains that the chmod won't be kept over reboots and that the following is needed to reset it after a reboot

nano /etc/udev/rules.d/99-intel-chmod666.rules

and put the following in it

KERNEL=="renderD128", MODE="0666"

Now I also have the I965 drivers installed on my proxmox host. I can't confirm if this is 100% needed so I would ask that people skip this next command and see if it works, if not then you can come back and install

On the Proxmox host console
apt install vainfo
This should install the required drivers and once installed running vainfo should return something like this

That's it, at this point you should the iGPU working in the CT. If you don't need network shares then you can move onto installing Frigate!

4) Passing through the network share

This part was taken from the this forum post
The post does a good job of explaining the issue of mounting a share in an unpriv container and how we are going to get around that in this instance.

Start the CT

Open the console to the LXC and do the following

Create the group "lxc_shares" with GID=10000 in the LXC which will match the GID=110000 on the PVE host.
groupadd -g 10000 lxc_shares
Add the user(s) that need access to the CIFS share to the group "lxc_shares".
(Frigate uses root as the user)
usermod -aG lxc_shares root
Shutdown the LXC

On Proxmox console
Create the mount point on the PVE host (this can be whatever you prefer)
mkdir -p /mnt/lxc_shares/frigate_network_storage

Add NAS CIFS share to /etc/fstab
nano /etc/fstab

!!!You will need to edit this line to include your own NAS path, Username and Password!!!
//IP_ADDRESS/SHARENAME /mnt/lxc_shares/frigate_network_storage cifs _netdev,noserverino,x-systemd.automount,noatime,uid=100000,gid=110000,dir_mode=0770,file_mode=0770,username=USERNAME,password=PASSWORD 0 0

This will then mount the share on startup to the Proxmox host

To mount the share NOW to the host type
mount /mnt/lxc_shares/frigate_network_storage

Type the following into the Proxmox console

cd /etc/pve/lxc
ls

You should now be in the lxc folder and see a list of CT conf files. We need to edit the CT conf file that corresponds to the CT we made above, you can find this in Proxmox

So I need to then type nano 114.conf to edit the conf file for the CT I am working on - YOURS MAY DIFFER

I then need to add the following to my conf file - YOURS MAY DIFFER DEPENDING ON THE PATH YOU CHOSE FOR THE FOLDER
mp0: /mnt/lxc_shares/frigate_network_storage/,mp=/opt/frigate/media

At this point you should now have some network storage mounted to the LXC container! If you followed along with everything else then you should also have a USB Coral and iGPU connected to the LXC container!

5) Installing Frigate

I'm just going to copy paste from my blog post about how to install frigate and get it configured as if you know then you can skip this part but if this is your first time trying then it might help you get it up and running. Inspiration for this blog post came from this blog post

Start the container within Proxmox and open the 'Console' to the container

Once at the command prompt let's do some updates

apt-get update
apt-get upgrade

Now lets get docker installed

apt install curl
curl -fsSL https://get.docker.com -o get-docker.sh
sh get-docker.sh

Once the above finishes, you can test Docker by running this command

docker run hello-world

I personally use and love portainer so we will be installing that to manage docker and the frigate container - if you wish to do this in the command line then that is fine but the instructions from here will be assuming you are using Portainer

docker run -d \
--name="portainer" \
--restart on-failure \
-p 9000:9000 \
-p 8000:8000 \
-v /var/run/docker.sock:/var/run/docker.sock \
-v portainer_data:/data \
portainer/portainer-ce:latest

Now we can open up another browser tab and head over to the Portainer GUI, the address is the same as your container but ending in :9000. In my example this is 192.168.1.59:9000 - You can get your IP address by typing ip addr

If all went well, you should be greeted with the below new installation screen, create a new password and note it down.

Once in Portainer, navigate around and check out the Containers List. Here you will see Portainer itself and the hello world test we ran earlier, here is where you will see all future Docker containers like Frigate once installed, feel free to delete the hello world container.

We now have installed Docker running in LXC Linux under Proxmox with management via a web based GUI.

Installing Frigate (Pre Reqs)
First we need to setup the folder structure that we will use for Frigate, this is how I set mine up, you are welcome to use your own folder structure

On the container console type the following:

cd /opt
mkdir frigate
cd frigate
mkdir media
mkdir storage

We are going to use the media folder for storing clips and recordings
We are going to use the storage folder for the db

While we are here we are going to make a basic config.yml file - this file contains all the settings for Frigate and you will spend a bit of time in here editing this file to suit your own setup

make sure you are in /opt/frigate and type nano config.yml

Edit the following in a word editor so that it suits your setup. Pay attention to the MQTT section and the Camera Section! Also if you have an older CPU (<10th gen) then pay attention to the HW Accel section! Paste the edited config into the config.yml file

database:
    path: /db/frigate.db
mqtt: # If you have a MQTT server then update this section, if not delete it.
    host: IP ADDRESS OF MQTT SERVER
    user: MQTT USERNAME
    password: MQTT PASSWORD
detectors:
    coral:
        type: edgetpu
        device: usb
ffmpeg:
    output_args:
        record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy
    hwaccel_args: # This is the section for the H/W Accelaration, you will need to figure out what settings you need to enable - https://docs.frigate.video/configuration/hardware_acceleration
        -c:v h264_qsv
       # - -hwaccel
       # - vaapi
       # - -hwaccel_device
       # - /dev/dri/renderD128
       # - -hwaccel_output_format
       # - yuv420p
objects: # Object to detect - https://docs.frigate.video/configuration/objects
    track:
        - person
record: # how long to record before and after motion
    events:
        pre_capture: 7
        post_capture: 10
cameras:
    CAMERA_NAME: # <------ Name the camera
        ffmpeg:
            inputs:
                - path: rtsp:https://USERNAME:PASSWORD@IPADDRESS:PORT/URL/FOR/STREAM/ # Update with your camera info (Normally the lower res stream, 1280*720 would be a good detection res)
                  roles:
                    - detect
                    - rtmp
                - path: rtsp:https://USERNAME:PASSWORD@IPADDRESS:PORT/URL/FOR/STREAM/ # Update with your camera info (Normally the higher res stream, used for actual recording)
                  roles:
                    - record
                   # - detect
                    #- rtmp
        rtmp:
            enabled: False # <-- RTMP should be disabled if your stream is not H264
        detect:
            width: 1280 # <---- update for your camera's resolution
            height: 720 # <---- update for your camera's resolution
            #width: 2688
            #height: 1520
        record: # what you want Frigate to record
            enabled: True
           # retain: # uncomment this section if you want Frigate to record this camera 24/7. This can use a lot of space, you have been warned!
           #     days: 1
           #     mode: all
            events: # this is the default recording mode where events are recorded, the number of days it saves events for is 10.
                retain:
                    mode: active_objects
                    default: 10
        snapshots:
            enabled: True
            retain:
                default: 10

ctrl-x to save the file

Now we have a config file located in /opt/frigate/config.yml

Installing the Frigate Docker container
Go back to Portainer and click on 'Stacks' --> Add Stack

Give the stack a name and then copy the below into the text box

version: "3.9"
services:
  frigate:
    container_name: frigate
    privileged: true # this may not be necessary for all setups
    restart: unless-stopped
    image: ghcr.io/blakeblackshear/frigate:stable
    shm_size: "64mb" # update for your cameras based on calculation above
    devices:
      - /dev/bus/usb:/dev/bus/usb # passes the USB Coral, needs to be modified for other versions
      - /dev/apex_0:/dev/apex_0 # passes a PCIe Coral, follow driver instructions here https://coral.ai/docs/m2/get-started/#2a-on-linux
      - /dev/dri/renderD128 # for intel hwaccel, needs to be updated for your hardware
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /path/to/your/config.yml:/config/config.yml
      - /path/to/your/storage:/media/frigate
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "5000:5000"
      - "8554:8554" # RTSP feeds
      - "8555:8555/tcp" # WebRTC over tcp
      - "8555:8555/udp" # WebRTC over udp
    environment:
      FRIGATE_RTSP_PASSWORD: "password"

Edit the RTSP Password and hit Deploy

At this point you should now have Frigate up and running. You can go back to the Containers section in Portainer and see the Frigate container running. From here you can open the logs and check that you have no errors.

If you now go to the container http:https://IPADDRESS:5000 then you should be greeted with your new Frigate container!

That's it... I have followed along as I wrote the doc/guide and I now have Frigate running in a LXC with a Coral, iGPU and network share

I'd welcome any feedback or suggestions. I **THINK ** this is more secure than running in a full priv LXC CT but welcome advise on that.

[akmolina28]

Good work figuring this out, especially the iGPU hardware acceleration! I starting going down the same rabbit hole trying to get hw acceleration working on my bare metal install on debian with a core i9-12900H and never got it working.

Eventually I tried out the beta version of Frigate and everything worked as it should. My best guess was that the frigate 0.11 image was built before these newer intel chips were fully supported.

If you continue to have issues, try out the beta. I have since migrated from bare metal to proxmox and I didn't have to do anything special to pass through the iGPU, I just looked at a few examples from the other discussions here to get it set up. Even H265 is working.

[cliffkujala]

Hello @akmolina28 Would you mind sharing your ProxMox LXC .conf file, frigate.yml and docker-compose setup to get the Frigate beta version running on the i9-12900H based ProxMox install? I'm running ProxMox on i5-12400 so I imagine my needed setup will be quite close to yours.

[Dougiebabe]

Not sure if this will help you @cliffkujala as this is for my UnPriv LXC as documented above but this is the full LXC.conf file for an unpriv LXC running the beta version on Intel Gold 7505 which is an 11th gen part

arch: amd64
cores: 2
features: nesting=1
hostname: frigate-unpriv-turnkey
memory: 2048
net0: name=eth0,bridge=vmbr0,firewall=1,hwaddr=8E:12:C8:69:34:7D,ip=dhcp,type=veth
ostype: debian
rootfs: local-lvm:vm-110-disk-0,size=20G
swap: 2048
unprivileged: 1
usb0: host=1a6e:089a,usb3=1
usb1: host=18d1:9302,usb3=1
lxc.cgroup2.devices.allow: c 189:* rwm
lxc.mount.entry: /dev/bus/usb/004 dev/bus/usb/004 none bind,optional,create=dir
lxc.cgroup2.devices.allow: c 226:0 rwm
lxc.cgroup2.devices.allow: c 226:128 rwm
lxc.mount.entry: /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,create=file 0, 0
lxc.mount.entry: /dev/dri dev/dri none bind,optional,create=dir

[akmolina28]

Hello @akmolina28 Would you mind sharing your ProxMox LXC .conf file, frigate.yml and docker-compose setup to get the Frigate beta version running on the i9-12900H based ProxMox install? I'm running ProxMox on i5-12400 so I imagine my needed setup will be quite close to yours.

arch: amd64
cores: 8
features: nesting=1
hostname: frigate
memory: 8192
net0: name=eth0,bridge=vmbr0,firewall=1,hwaddr=AE:F8:58:3C:9D:00,ip=dhcp,type=veth
ostype: debian
rootfs: local-lvm:vm-103-disk-0,size=128G
swap: 1024
usb0: host=1a6e:089a,usb3=1
usb1: host=18d1:9302,usb3=1
lxc.cgroup2.devices.allow: c 189:* rwm
lxc.mount.entry: /dev/bus/usb/001 dev/bus/usb/001 none bind,optional,create=dir
lxc.cgroup2.devices.allow: c 226:0 rwm
lxc.cgroup2.devices.allow: c 226:128 rwm
lxc.mount.entry: /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,create=file 0, 0
lxc.mount.entry: /dev/dri dev/dri none bind,optional,create=dir

[ivlis]

Hi @Dougiebabe and @akmolina28

Have you tried running 0.12.0 in an unprivileged LXC? I'm getting this error:

frigate-frigate-1  | chown: changing ownership of '/dev/shm/logs/frigate': Invalid argument
frigate-frigate-1  | chown: changing ownership of '/dev/shm/logs/go2rtc': Invalid argument
frigate-frigate-1  | chown: changing ownership of '/dev/shm/logs/nginx': Invalid argument

For some reason, see #6075 0.12 needs to change the ownership of these files to nobody, and it can't do that in an unprivileged environment.

[Dougiebabe]

Proxmox info:

[ivlis]

I'm still on 7.2, maybe that's the issue. Thank you, I will try updating my pve.

[ivlis]

There is a bug in Debian (or proxmox's debian container), because I cannot change the ownership in a Debian lxc container, but I can in Ubuntu. Both OSes use the same uid/guid for nobody/nogroup so LXC is ruled out. So I should be okay switching to Ubuntu 22.10 as an underlying lxc container. I will update the ticket is that works.

[FrigateEnthusiast]

There is a bug in Debian (or proxmox's debian container), because I cannot change the ownership in a Debian lxc container, but I can in Ubuntu. Both OSes use the same uid/guid for nobody/nogroup so LXC is ruled out. So I should be okay switching to Ubuntu 22.10 as an underlying lxc container. I will update the ticket is that works.

I have Frigate 12 running in Docker based on a Proxmox 7.4 Debian LXC container.
Here's my container config: Works for both a usb and pcie Coral.

arch: amd64
cores: 6
features: nesting=1
hostname: frigate
memory: 4096
net0: name=eth0,bridge=vmbr0,hwaddr=76:8E:AC:DD:91:E9,ip=dhcp,type=veth
onboot: 1
ostype: debian
parent: InitialInstall
rootfs: pooldino:subvol-100-disk-0,size=3800G
swap: 4096
lxc.cgroup2.devices.allow: c 226:0 rwm
lxc.cgroup2.devices.allow: c 226:128 rwm
lxc.cgroup2.devices.allow: c 29:0 rwm
lxc.cgroup2.devices.allow: c 189:* rwm # usb
lxc.apparmor.profile: unconfined
lxc.cgroup2.devices.allow: a
lxc.mount.entry: /dev/bus/usb dev/bus/usb none bind,optional,create=dir
lxc.mount.entry: /dev/apex_0 dev/apex_0 none bind,optioanl,create=file
lxc.cap.drop:
lxc.mount.auto: cgroup:rw

[ivlis]

Thanks, for sharing @FrigateEnthusiast , I realized there was a mistake in my uid mapping: #6075

[SilverHazeLazers]

Thanks for the guide @Dougiebabe, would you happen to know if it is possible to store recordings on the host?

I have tried mapping the group on the LXC (media) to a group on the host (media) to no avail.

I think my problem is that the root user on the LXC used by Frigate cant be mapped to my user 'media' on the host.

Any suggestions would be much appreciated

[FrigateEnthusiast]

Reply below meant for you.

[FrigateEnthusiast]

I created a big ZFS pool in Proxmox. In that pool (pooldino - dinosaur spinning drives) was the root volume for my frigate container.
So on the host, I can see /pooldino/subvol-100-disk-0 (container root vol) mounted, and in it I can access ./opt/frigate/config.yml, and directories media and storage...

[noloarmi]

Thanks for this guide. I followed it carefully and got really far with my Frigate journey. I'm really new to Frigate and Home assistant world and this is exactly what I needed.

However, after following the steps for iGPU passthrough, it looks like it is working. I can even confirm that the openvino install worked as my interference speed is like 12ms (vs CPU which is like 280ms). This is really good so far.

Moving on .. after adding the line below in config.yml camera section.

hwaccel_args: preset-vaapi

It does seem to work but Im getting an error (screenshot below) from the frigate system page

In the logs I see this

frigate.util ERROR : Unable to poll intel GPU stats: Failed to initialize PMU! (Permission denied)

Like I mentioned, Im really new to this and so far I'm it look me 4 days just to have a decent config.yml that works well (still need more improvements!).

Is this expected behavior? Can it be safely ignored?

Is there a way to check what this error impacts in terms of hardware acceleration of the LXC container?

TIA!

[NickM-27]

all it means is that the stats can not be retrieved, there is no actual issue with the hwaccel itself

[Dougiebabe]

I installed intel_gpu_top on the proxmox host and confirmed that the igpu is being used there. It would be great to see the stats in Frigate but I haven't the time or the drive to try and get it working. I assume it's a permissions issues within docker maybe...

[metalrise]

my little contribution for google search results, frigate as LXC in Proxmox, just running
intel_gpu_top Failed to detect engines! (No such file or directory) (Kernel 4.16 or newer is required for i915 PMU support.) Segmentation fault

but running, shows the actual stats
intel_gpu_top -d sriov

still no solution for
frigate.util.services ERROR : Unable to poll intel GPU stats: Failed to detect engines! (No such file or directory) (Kernel 4.16 or newer is required for i915 PMU support.)

[keplenk]

Hello, I've used this guide and it worked well with iGPU detectors. Now I just received a M.2 pcie which I confirmed is detected by the proxmox host.

root@proxmox:/etc/default# lspci -nnv | grep 089a
01:00.0 System peripheral [0880]: Global Unichip Corp. Coral Edge TPU [1ac1:089a] (prog-if ff)
        Subsystem: Global Unichip Corp. Coral Edge TPU [1ac1:089a]

Looks like this guide only tackles USB Corals vs M.2 Pcie ones.

Anyone knows any guide for Proxmox LXC with M.2 pcie coral passthrough?

Still new and learning these stuff. Thank you1

[FrigateEnthusiast]

See my config posted above.

"works for both a usb and pcie Coral."

The one additional step over getting a USB Coral working is installing the Coral pcie driver. Go here:

https://coral.ai/docs/m2/get-started

Tricky parts MAY BE getting the driver to recompile if it senses your kernel version is different than the canned binary. You would have to install the kernel headers used by proxmox. Debian generic won't cut it.

Some "blacklist" the coral driver to prevent it from loading except by the frigate image. I don't bother with that since this is a dedicated cam/frigate box so just let the driver load at the promox level and pass in the /dev/apex_NN dev(s). Simpler.

So give that a whiz... and post back of any specific problems. Easier to give guidance for a specific fail/error than write an encyclopedia here... lol

Quick steps for you:

1. You may need to setup iommu like this:
   Coral PCIe TPU in Proxmox via VM ( Debian ) without HW Accel? #6403 (comment)

2. promox host - install Coral drivers.

3. ls /dev/apex_* - If the dev shows you are good to go to edit the container and docker configs for the pcie dev.

4. If you arrived here - get happy.

[Bettman66]

Coral Dual Edge TPU
Proxmox Host:
apt install pve-headers-5.15.104-1-pve
or your current kernel header
sh -c "echo 'SUBSYSTEM==\"apex\", MODE=\"0755\", GROUP=\"apex\"' >> /etc/udev/rules.d/65-apex.rules"

curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
echo "deb https://packages.cloud.google.com/apt coral-edgetpu-stable main" | tee /etc/apt/sources.list.d/coral-edgetpu.list

apt update
apt install gasket-dkms libedgetpu1-std
nano /etc/pve/lxc/***.conf
insert this:
lxc.cgroup2.devices.allow: c 120:0 rwm
lxc.mount.entry: /dev/apex_0 dev/apex_0 none bind,optional,create=file
lxc.mount.entry: /dev/apex_1 dev/apex_1 none bind,optional,create=file

[zaidranger]

The igpu passthrough works with idmap, i also encountered a simillar issue where my guest would show the render file mapped with group render and vainfo works but no apps/services could have access to it, the problem was because render file needs to be mounted with it's directory after mounting that it works without any issues, hope this helps.

Also oddly enough, i found many guide on igpu passthrough on priviledged container only mounting the render file without the dir, and i remembered it didn't need it before, don't know when things changed but i can confirm it works by mounting both the dir and render file on lxc conf.

[Dougiebabe]

I would be interested in how you did this, like I said in the guide I had to pass the whole igpu through as I couldn't get the idmaps to work.

[zaidranger]

Sorry, my previous post must have been a bit confusing, what i did was simply mounting both the folder "/dev/dri" and the render file "/dev/dri/render12X" in lxc config, then proceed to do the idmaps as usual. I noticed by only mounting the render file without also mounting the "dri" folder the lxc would boot with the render files and mapped to the correct group and user, but none of my services could get access to it, after mouting it with the "dri" folder everything works, though like i said above, i didn't need to do this before in previous version if i remembered correctly, but you can see other old guides about idmap them not mounting the folder also so i guess something must've changed.

[Dougiebabe]

I've updated the guide slightly as there was a mistake in the NAS Storage part where the group didn't actually have access to the NAS storage mounted to the host, the fix is in the fstab part. I had been using local storage in my own but have now swapped over to NAS storage for the recordings.

I have also updated the docker compose part to the default docker compose as I had missed the newer go2rtc parts for the newer v12 version.

Everything has been running rock solid for me since moving this over to the unprivileged LXC and I am also able to use the GPU for Plex transcoding on another LXC which wasn't possible when using VMs.

[EJones86]

First off, thanks for the great guide! But...

I have been at this all day, and I absolutely cannot get the Coral USB to be recognised by Frigate unless I switch to a privileged container.

The trouble is if I do that it breaks everything else Docker related including Portainer (agent).

Fairly new to all this (5 days!?) but I have managed to get a few other VMs running in Proxmox, including OMV, Plex, Sonarr, Radarr, SABnzbd etc, and Frigate running fine with hardware decoding, it just WILL NOT find the Coral in an unprivileged container!!

What info do you need!? TIA

[EJones86]

P.s. I did try several times on Debian too, thought i'd give the Ubuntu image a go on this last pass...

[Dougiebabe]

Yeah I mean everything looks right... the only thing I noticed is my LXC is called frigate-unpriv-turnkey so I assume I must be running Debian turnkey instead of Ubuntu.

Maybe give that a try although I'm not sure why I'm using turnkey and not Ubutnu

[EJones86]

Yes i saw Ubuntu in the conf above hence my p.s comment, but then I realised your guide says Ubuntu anyway! I think another guide I saw was using turnkey so maybe you saw that one too!

How frustrating!

[Dougiebabe]

did it work with the turnkey version?

[EJones86]

I have just finished setting it all up on turnkey, and no, it didn't work, BUT...

I tried changing the container to privileged just to get the Coral recognised for now, and everything else is still working!! Docker still responding, the Portainer agent is running, and Frigate is inferinginging itself at <8ms - even detected my wife leaving the house, albeit with a 62% probability that she was a dog :')

[NorthernMunkey]

Very good guide. one issue I keep having is every time i reboot the proxmox server, the usb assignment for the coral device changes bus assigment usually between bus 004 and 003
Bus 003 Device 002: ID 18d1:9302 Google Inc.
Bus 004 Device 002: ID 18d1:9302 Google Inc.

which means i have to check and edit the conf file for the passthrough

does anyone get the same and if so what do you do to combat it?

[EJones86]

Have you passed through the entire port?

[FrigateEnthusiast]

If you pass in the entire usb bus, then it doesn't matter if it changes.

lxc.mount.entry: /dev/bus/usb dev/bus/usb none bind,optional,create=dir

detectors:
coralusb:
type: edgetpu
device: usb

[NorthernMunkey]

thanks, will that effect if other devices are plugged into the usb for other machines e.g zigbee gateway

[FrigateEnthusiast]

In your example (zigbee), only if you had zigbee drivers that are loading in each "space" and trying to find/setup. That is usually solved by blacklisting the device.

[Photo64]

I have been trying for days to passthru my coral tpu but I haven't been successful so I was hoping maybe someone might know where I am going wrong. I followed the steps above but I still get the message that it can't find the edge tpu.

This is my current setup for the id.conf file. I have tried multiple entries based on information given above but I am still unsuccessful in getting it to pass thru. Any ideas would be welcomed!

arch: amd64
cores: 2
features: nesting=1
hostname: frigate
memory: 2048
mp0: /mnt/lxc_shares/frigate_network_storage/,mp=/opt/frigate/media
net0: name=eth0,bridge=vmbr0,firewall=1,hwaddr=BA:9A:B3:73:71:91,ip=dhcp,type=>
ostype: ubuntu
rootfs: nvme3:vm-116-disk-0,size=20G
swap: 2048
unprivileged: 1
usb0: host=1a6e:089a,usb3=1
usb1: host=18d1:9302,usb3=1
lxc.cgroup2.devices.allow: c 189:* rwm
lxc.mount.entry: /dev/bus/usb/006 dev/bus/usb/006 none bind,optional,create=dir
lxc.cgroup2.devices.allow: c 226:0 rwm
lxc.cgroup2.devices.allow: c 226:128 rwm
lxc.mount.entry: /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,cre>
lxc.mount.entry: /dev/dri dev/dri none bind,optional,create=dir

[FrigateEnthusiast]

Trying to run unprivileged has given many nothing but "fail". I've never tried it. It that absolutely needed?

[Photo64]

I just tried removing that line from the conf file because I had seen some say removing it worked for them however now I can't start up docker anymore.I am getting these errors that I can't figure out how to get past to start docker again
-> docker.service: Start request repeated too quickly.
-> docker.service: Failed with result 'exit-code'.

If in trying to run as privileged would I need now to start all over and create a container with that unchecked or what would you recommend?

[FrigateEnthusiast]

Sorry, I'm a docker noob, so can't help on that.

[Photo64]

Docker issue aside, would it be preferred to follow the directions above to create a new lxc with the 'unprivileged' setting unchecked and then reapply all the remaining steps?

Better yet, is there another set of instructions for setting this as a privileged container that you might be aware that I could follow?

[Photo64]

Well I'll be . . . for the heck of it I put the unprivileged line back in but instead of making it unprivileged: 1 I made it unprivileged: 0 and suddenly docker came back up and my coral usb tpu is working in Frigate. Phew, that was painful but I'm glad to see it finally working now. Thanks for the guidance.

[Photo64]

I have one more issue that is perplexing me and that is to store my Frigate recordings on a shared drive. I have been unable to get this to work and I am not sure if it's due to incorrect setup or possibly a permissions issue.

Following the steps above I added this line to my fstab file -> //192.168.1.110/frigate_network_storage /mnt/lxc_shares/frigate_network_storage cifs _netdev,noserverino,x-systemd.automount,noatime,uid=100000,gid=110000,dir_mode=0770,file_mode=0770,username=xxxxxx><frigate,password=xxxxxxxx 0 0

I have this in my container conf file -> mp0: /mnt/lxc_shares/frigate_network_storage/,mp=/opt/frigate/media

I added this volume to the portainer stack -> - /opt/frigate/media:/media/frigate

in the frigate.yml file I added this so that the database is local and not on the network storage

  path: /opt/frigate/storage/frigate.db

When I did all this Frigate appeared to be running within docker however I couldn't get the UI page to come up it just gave me a spinning circle.

Anyone with more docker/proxmox knowledge than I is there a way to determine if my config setup is incorrect or a possible permissions error somewhere? Any guidance or ideas would be appreciated.

[Photo64]

I am actually running Privileged now as that was the only way I could get the TPU to pass thru

[Photo64]

In my attempts I actually see this in the lxc with a bunch of .mp4 files however I can't see them on my nas -> root@frigate:/mnt/lxc_shares/frigate_network_storage/recordings/2023-07-10/14/driveway# ls
10.05.mp4 11.28.mp4 12.28.mp4 13.28.mp4 14.27.mp4 15.27.mp4 16.27.mp4 17.27.mp4 18.27.mp4

Does this mean it is not mounting?

**edit -> When I execute "mount -a" I don't receive any errors so I was assuming that it was mounting but maybe it's not??

[Dougiebabe]

Yeah it's not mounting and saving to the lxc

Tey removing uid and gid part from fstab as they are if you run unprivileged

[Photo64]

I removed both uid and gid and ran mount -a again but still not seeing anything in my nas folder.

[Dougiebabe]

//192.168.1.110/frigate_network_storage /mnt/lxc_shares/frigate_network_storage cifs _netdev,noserverino,x-systemd.automount,noatime,uid=100000,gid=110000,dir_mode=0770,file_mode=0770,username=xxxxxx><frigate,password=xxxxxxxx 0 0

After username you have a ><frigate part, that doesn't look right. Is that in your fstab or an error with copy paste?

[Dougiebabe]

//192.168.1.110/frigate_network_storage /mnt/lxc_shares/frigate_network_storage cifs _netdev,noserverino,x-systemd.automount,noatime,uid=100000,gid=110000,dir_mode=0770,file_mode=0770,username=xxxxxx><frigate,password=xxxxxxxx 0 0

After username you have a ><frigate part, that doesn't look right. Is that in your fstab or an error with copy paste?

[Photo64]

Sorry, I just noticed that too but it's a copy/paste error, here is the correct syntax
//192.168.1.110/frigate_network_storage /mnt/lxc_shares/frigate_network_storage cifs _netdev,noserverino,x-systemd.automount,noatime,dir_mode=0770,file_mode=0770,username=xxxxxxx,password=xxxxxxxx 0 0

[Photo64]

Still struggling to try and get my shared folder mounted however looking at this wouldn't it suggested that it is mounted?
root@pve1:~# df -h
Filesystem Size Used Avail Use% Mounted on
udev 31G 0 31G 0% /dev
tmpfs 6.2G 1.6M 6.2G 1% /run
/dev/mapper/pve-root 94G 25G 65G 28% /
tmpfs 31G 34M 31G 1% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
/dev/nvme0n1p2 1022M 344K 1022M 1% /boot/efi
/dev/fuse 128M 20K 128M 1% /etc/pve
192.168.1.110:/mnt/HomePool/pve1/pve1-backups 6.7T 76G 6.7T 2% /mnt/pve/pve1-backups
//192.168.1.110/frigate_network_storage 6.7T 140K 6.7T 1% /mnt/lxc_shares/frigate_network_storage
//192.168.1.110/frigate 6.7T 140K 6.7T 1% /mnt/lxc_shares/frigate_network_storage
tmpfs 6.2G 0 6.2G 0% /run/user/0

What I am having trouble understanding is the full path to the shared folder is //192.168.1.110/mnt/HomePool/Frigate/frigate_network_storage. How does it know that //192.168.1.110/frigate_network_storage. would map to that? What would I need to change to make that work? I tried entering the full path but it didn't like that and gave me an error message.

[Dougiebabe]

I'm in between computers atm and been answering on my phone. Once I'm setup I'll send screen shots of my setup to see if it helps.

I mounted the shares on the host as to pass through to an unprivileged lxc thats how I understood it needed to be done.

With a privileged lxc you may not need to do that and might be able to mount directly in the lxc itself.

If your going to use a privileged lxc then I think there is a different guide you can follow, not sure if it covers connecting to shared/nas storage though.

Hopefully have my pc sorted this evening

[Yonny24]

Great! This will come in useful for me in a week. Been running Frigate on lxc proxmox for a while now as a backup to BI.
But without any coral usb, just on cpu.
I'll need to passthrough and never done it before.

By any chance do we know when Frigate+ docker images will be released?
I've started uploading some of my images to their + dashboard

Thanks!

[matt1131]

I'm using this to try and get HW acceleration working. I followed the part to pass through my iGPU to Frigate running in HAOS and I get to running vainfo and get the following:

Lost on where to go from here. I'm using a NUC with i7-1165G7 Processor

[daad919]

Thanks for the reply!

[pzerosix]

I had similar errors with my 12th gen Intel iGPU. You might need to install the Intel Media Driver for VAAPI.
You can check out this link for reference. https://github.com/intel/media-driver.git
I am a newb, so please proceed at your own risk,
That being said, I was able to get it to work with lots of googling and help from everyone :)

All of this will be from your proxmox terminal.

1. First install LibVA

• Install common packages

sudo apt-get install git cmake pkg-config meson libdrm-dev automake libtool

• Get latest libva

git clone https://github.com/intel/libva.git
cd libva

• Build with autogen

./autogen.sh --prefix=/usr --libdir=/usr/lib/x86_64-linux-gnu
make
sudo make install

2. Build and install GmmLib

• go to root directory

git clone https://github.com/intel/gmmlib.git
cd gmmlib/
mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Release -DARCH=64 ..
make 
sudo make install
cd ../..

3. Install Intel Media Driver

git clone https://github.com/intel/media-driver.git
mkdir build_media
cd build_media
cmake ../media-driver
make 
sudo make install
cd ..

4. Verify

vainfo

If everything went well, your vainfo should now show all the supported profiles properly.
Good luck!

[matt1131]

Yes, this worked, thank you!

However, it looks like this post may be a dead end for me. I have HAOS running in a Proxmox VM with Frigate as an add-on (leftover from before I migrated to Proxmox) and thought this might help get hwaccel working. No luck so far and I'm not savvy enough to know what I'd need to modify in these instructions to apply to my setup.

[daad919]

Thanks for the info, i've got my problem solved with following commands:

apt install vainfo

sudo apt-get install --reinstall intel-media-va-driver:amd64

apt install autoconf libtool libdrm-dev xorg xorg-dev openbox libx11-dev libgl1-mesa-glx

[pzerosix]

Glad this helped out.
I don't have any experience with HAOS, so I don't know if this is applicable to you, but you can try running Home Assistant in docker instead.
After following the guide above, I also got Home Assistant docker, installed HACS and have the Frigate card installed.
I just got things working recently, so I have to see if things are stable, but so far everything seems to be working.

[SofaSurfer76]

Hi,
I have a problem saving my clips on network SMB shared folder, I have followed the network setup of your guide, my folder is mounted and accessible from docker console but I cannot save my clips/recordings on it.
I have tried to follow this official guide without success:
https://pve.proxmox.com/wiki/Unprivileged_LXC_containers
my folder stays "nouser" and "nogroup" and I have "permission denied" if I try to modify something. From frigate logs I have a similar message.

root@docker ~# ls -l /mnt/pve/omv/
total 0
drwxr-xr-x 2 nobody nogroup 0 Sep 16 10:33 Media
drwxr-xr-x 2 nobody nogroup 0 Sep  1 19:40 frigate
drwxr-xr-x 2 nobody nogroup 0 Aug 14 12:51 images

Any suggestion?

[EDIT] SOLVED!

[nayzm]

Honestly, I can't thank you enough for this very helpful guide. It fixed every issue I had stumbled upon, much appreciated. 👍🏼

[yodaphone]

Hi.

I'm new to frigate and just installed it on Proxmox 8.1 in a LXC container with docker

Everything works fine & I'm able to see the camera etc.

BUT.... Once i enable recording I'm getting this error message & I'm unable to record

2024-02-08 21:33:38.887967722  [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f16bc01f340] moov atom not found
2024-02-08 21:33:38.888058174  [ERROR:[email protected]] global cap.cpp:164 open VIDEOIO(CV_IMAGES): raised OpenCV exception:
2024-02-08 21:33:38.888060420  
2024-02-08 21:33:38.888062157  OpenCV(4.7.0) /io/opencv/modules/videoio/src/cap_images.cpp:267: error: (-215:Assertion failed) number < max_number in function 'icvExtractPattern'
2024-02-08 21:33:38.888063262  
2024-02-08 21:33:38.888064271  
2024-02-08 21:33:38.905060463  [2024-02-08 21:33:38] frigate.record.maintainer      WARNING : Failed to probe corrupt segment /tmp/cache/FrontCam@20240208213336+0530.mp4
2024-02-08 21:33:38.905670407  [2024-02-08 21:33:38] frigate.record.maintainer      WARNING : Discarding a corrupt recording segment: /tmp/cache/FrontCam@20240208213336+0530.mp4
2024-02-08 21:33:44.104371158  [2024-02-08 21:33:44] ffmpeg.FrontCam.record         ERROR   : [mp4 @ 0x55fe106f2340] Could not find tag for codec pcm_mulaw in stream #1, codec not currently supported in container
2024-02-08 21:33:44.104527214  [2024-02-08 21:33:44] ffmpeg.FrontCam.record         ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument
2024-02-08 21:33:44.104598074  [2024-02-08 21:33:44] ffmpeg.FrontCam.record         ERROR   : Error initializing output stream 0:1 -- 
2024-02-08 21:33:44.104610202  [2024-02-08 21:33:44] ffmpeg.FrontCam.record         ERROR   : 
2024-02-08 21:33:44.104724969  [2024-02-08 21:33:44] watchdog.FrontCam              INFO    : Terminating the existing ffmpeg process...
2024-02-08 21:33:44.104727618  [2024-02-08 21:33:44] watchdog.FrontCam              INFO    : Waiting for ffmpeg to exit gracefully...
s6-rc: info: service legacy-services: stopping
s6-rc: info: service legacy-services successfully stopped
s6-rc: info: service nginx: stopping
s6-rc: info: service go2rtc-healthcheck: stopping

I'm running Proxmox on a Dell optiplex with Intel(R) Core(TM) i5-6500 CPU @ 3.20GHz

My camera is Dahua compatible (Asecam). its a 2K IP POE Camera

This is the GPU that i have

24: PCI 02.0: 0300 VGA compatible controller (VGA)
  [Created at pci.386]
  Unique ID: _Znp.jWo7VreMzP0
  SysFS ID: /devices/pci0000:00/0000:00:02.0
  SysFS BusID: 0000:00:02.0
  Hardware Class: graphics card
  Model: "Intel HD Graphics 530"
  Vendor: pci 0x8086 "Intel Corporation"
  Device: pci 0x1912 "HD Graphics 530"
  SubVendor: pci 0x1028 "Dell"
  SubDevice: pci 0x06b9
  Revision: 0x06
  Driver: "i915"
  Driver Modules: "i915"
  Memory Range: 0xf6000000-0xf6ffffff (rw,non-prefetchable)
  Memory Range: 0xe0000000-0xefffffff (ro,non-prefetchable)
  I/O Ports: 0xf000-0xf03f (rw)
  Memory Range: 0x000c0000-0x000dffff (rw,non-prefetchable,disabled)
  IRQ: 180 (413888 events)
  Module Alias: "pci:v00008086d00001912sv00001028sd000006B9bc03sc00i00"
  Config Status: cfg=new, avail=yes, need=no, active=unknown

I 'm able to see the same display info inside the container as well.

This is my config.yml

mqtt:
    host: 192.168.2.25
    user: user
    password: password
ffmpeg:
  output_args:
      record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy
  hwaccel_args: preset-vaapi
detectors:
  coral1:
    type: edgetpu
    device: pci:0
  coral2:
    type: edgetpu
    device: pci:1
#Global Object Settings
objects:
  track:
    - person
    - car
  filters:
    person:
      min_area: 5000
      max_area: 100000
record: # how long to record before and after motion
    events:
        pre_capture: 7
        post_capture: 10
birdseye:
  enabled: True
  mode: continuous
cameras:
  FrontCam:
    ffmpeg:
      inputs:
        # High Resolution Stream
        - path: rtsp:https://user:[email protected]:554/cam/realmonitor?channel=1&subtype=00
          roles:
            - record
        # Low Resolution Stream
        - path: rtsp:https://user:[email protected]:554/cam/realmonitor?channel=2&subtype=00
          roles:
            - detect
    record:
     enabled: True
     retain:
      days: 3
      mode: all
     events:
      retain:
       default: 30
       mode: motion 
    detect:
      width: 3840
      height: 2160

this is my docker compose

version: "3.9"
services:
  frigate:
    container_name: frigate
    privileged: true # this may not be necessary for all setups
    restart: unless-stopped
    image: ghcr.io/blakeblackshear/frigate:stable
    shm_size: "1024mb" # update for your cameras based on calculation above
    devices:
      - /dev/apex_0:/dev/apex_0 # passes the USB Coral, needs to be modified for other versions
      - /dev/apex_1:/dev/apex_1
      - /dev/dri/renderD128 # for intel hwaccel, needs to be updated for your hardware
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /opt/frigate/config.yml:/config/config.yml
      - /opt/media:/media/frigate
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "5000:5000"
      - "8554:8554" # RTSP feeds
      - "8555:8555/tcp" # WebRTC over tcp
      - "8555:8555/udp" # WebRTC over udp
    environment:
      FRIGATE_RTSP_PASSWORD: "somepass"

Any help in getting recording to work would be really helpful

[Zanadar]

I have this same issue but only after running for a few days!

[AussiSG]

Things got a lot easier

This is not needed anymore

lxc.cgroup2.devices.allow: c 226:0 rwm
lxc.cgroup2.devices.allow: c 226:128 rwm
lxc.mount.entry: /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,create=file 0, 0
lxc.mount.entry: /dev/dri dev/dri none bind,optional,create=dir

The following works right away,
44 and 105 are the group IDs of the group video (44 in my case) and render ( 105 in my case ) in the LXC container:

dev0: /dev/dri/card0,gid=44,uid=0
dev1: /dev/dri/renderD128,gid=105,uid=0

[mgmuk]

This works for me! I've been using the recommended approach with gid mappings + groups and mounts in an unprivileged container. Whilst vainfo and even ffmpeg seems to work correctly inside the lxc, it just wasn't working inside the frigate docker container. Your approach worked for me with a 13th generation NUC using vaapi although qsv doesn't work. I have noted however that this explicitly passes through the hardware device, so would mean you're not able to share with other lxcs.

[HimbeersaftLP]

I can confirm that this works for me using VA-API on an Intel iGPU as well

[woody4165]

Hi all

I've followed this guide (Thanks!) to install Frigate on Proxmox

What should I do to upgrade Frigate from 0.13.1 to 0.13.2 ?

AFAIK, I didn't used docker-compose

Thanks!

[vexersa]

I am not sure where I went wrong but I followed the guide verbatim but I get an error that the Frigate config file cannot be found:

2024-05-02 14:32:45.232393550  [INFO] Preparing Frigate...
2024-05-02 14:32:45.232418283  [ERROR] Frigate config file not found
s6-rc: info: service nginx successfully started
s6-rc: info: service legacy-services: starting
2024-05-02 14:32:45.234385800  [INFO] Preparing new go2rtc config...
2024-05-02 14:32:45.237195246  [INFO] Starting NGINX...
2024-05-02 14:32:45.238773486  [INFO] Service Frigate exited with code 1 (by signal 0)
s6-rc: info: service legacy-services successfully started
s6-rc: info: service legacy-services: stopping
s6-rc: info: service legacy-services successfully stopped
s6-rc: info: service nginx: stopping
s6-rc: info: service go2rtc-healthcheck: stopping
2024-05-02 14:32:45.248561615  [INFO] The go2rtc-healthcheck service exited with code 256 (by signal 15)
s6-rc: info: service go2rtc-healthcheck successfully stopped
2024-05-02 14:32:45.250572945  [INFO] Service NGINX exited with code 256 (by signal 15)
s6-rc: info: service nginx successfully stopped
s6-rc: info: service nginx-log: stopping
s6-rc: info: service frigate: stopping
s6-rc: info: service frigate successfully stopped
s6-rc: info: service go2rtc: stopping
s6-rc: info: service frigate-log: stopping
s6-rc: info: service nginx-log successfully stopped
s6-rc: info: service frigate-log successfully stopped
2024-05-02 14:32:45.258920984  [INFO] The go2rtc service exited with code 256 (by signal 15)
s6-rc: info: service go2rtc successfully stopped
s6-rc: info: service go2rtc-log: stopping
s6-rc: info: service go2rtc-log successfully stopped
s6-rc: info: service log-prepare: stopping
s6-rc: info: service s6rc-fdholder: stopping
s6-rc: info: service log-prepare successfully stopped
s6-rc: info: service legacy-cont-init: stopping
s6-rc: info: service s6rc-fdholder successfully stopped
s6-rc: info: service legacy-cont-init successfully stopped
s6-rc: info: service fix-attrs: stopping
s6-rc: info: service fix-attrs successfully stopped
s6-rc: info: service s6rc-oneshot-runner: stopping
s6-rc: info: service s6rc-oneshot-runner successfully stopped

My Portainer stack config file:

version: "3.9"
services:
  frigate:
    container_name: frigate
    privileged: true # this may not be necessary for all setups
    restart: unless-stopped
    image: ghcr.io/blakeblackshear/frigate:stable
    shm_size: "1024mb" # update for your cameras based on calculation above
    devices:
      - /dev/bus/usb:/dev/bus/usb # passes the USB Coral, needs to be modified for other versions
      - /dev/apex_0:/dev/apex_0 # passes a PCIe Coral, follow driver instructions here https://coral.ai/docs/m2/get-started/#2a-on-linux
      - /dev/dri/renderD128 # for intel hwaccel, needs to be updated for your hardware
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /opt/frigate/config.yml:/config/config.yml
      - /opt/frigate/media:/media/frigate
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "5000:5000"
      - "8554:8554" # RTSP feeds
      - "8555:8555/tcp" # WebRTC over tcp
      - "8555:8555/udp" # WebRTC over udp
    environment:
      FRIGATE_RTSP_PASSWORD: "somepass"

Config file (config.yml):

database:
    path: /storage/frigate.db
mqtt: # If you have a MQTT server then update this section, if not delete it.
    host: 192.168.0.13
    user: frigate-mqqt
    password: somepass
detectors:
    coral:
        type: edgetpu
        device: usb
ffmpeg:
    output_args:
        record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy
    hwaccel_args: # This is the section for the H/W Accelaration, you will need to figure out what settings you need to enable - https://docs.frigate.video/configuration/hardware_acceleration
       -hwaccel_args: preset-vaapi
       # -c:v h264_qsv
       # - -hwaccel
       # - vaapi
       # - -hwaccel_device
       # - /dev/dri/renderD128
       # - -hwaccel_output_format
       # - yuv420p
objects: # Object to detect - https://docs.frigate.video/configuration/objects
    track:
        - person
#        - bicycle
#        - cat
#        - dog
#        - car
#        - motorcycle
record: # how long to record before and after motion
    events:
        pre_capture: 7
        post_capture: 10
cameras:
    reolink_garage: # <------ Name the camera
        ffmpeg:
            inputs:
                - path: rtsp:https://user:[email protected]:554/h264Preview_01_sub # Update with your camera info (Normally the lower res stream, 1280*720 would be a good detection res)
                  roles:
                    - detect
                    - rtmp
                - path: rtsp:https://user:[email protected]:554/h264Preview_01_main # Update with your camera info (Normally the higher res stream, used for actual recording)
                  roles:
                    - record
                   # - detect
                    #- rtmp
        rtmp:
            enabled: True # <-- RTMP should be disabled if your stream is not H264
        detect:
            width: 1920 # <---- update for your camera's resolution
            height: 1080 # <---- update for your camera's resolution
            #width: 2688
            #height: 1520
        record: # what you want Frigate to record
            enabled: True
           # retain: # uncomment this section if you want Frigate to record this camera 24/7. This can use a lot of space, you have been warned!
           #     days: 1
           #     mode: all
            events: # this is the default recording mode where events are recorded, the number of days it saves events for is 10.
                retain:
                    mode: active_objects
                    default: 10
        snapshots:
            enabled: True
            retain:
                default: 10

[swiftbird07]

Are sure the config file is in the right place? Does cat /opt/frigate/config.yml output the config file?

[swiftbird07]

Did anybody try to pass-through an AMD APU (integrated GPU) using this method? Is it even possible?

[sohail579]

I have a Threadripper Pro which does not have an IGPU so do I pass nothing though and let software handle it?

[andregoncalvespires]

Hello everybody.
I've been running my Frigate installation on LXC Proxmox + coral USB and network storage for a while now. 45 days ago I upgraded my cameras by adding Reolink models. In total I use 10 cameras. I have noticed that LXC's memory usage has been behaving strangely, on an increasing curve and never decreasing. Strangely, there have been spikes in SWAP usage where the Frigate and Portainer crash and I can only reestablish it after restarting LXC.
I used 4096 Mb of ram for lxc, but I increased it to 8192 Mb. Swap was configured with 512 Mb and I also increased it to 1024 Mb. SHM is at 512 Mb, and according to the calculation in the documentation I would need something around 400 Mb. For tmpfs , I kept the 1000 Mb of documentation.

Could anyone comment or indicate if they have experienced something similar?

In this image, when I started LXC, the memory used was around 2 Gb

[FrigateEnthusiast]

So it looks like you have a classic memory leak.

Try running "top". When up and running press "M" and "R" to sort by memory size, largest "porkers" at the top. Your leaker should be the one growing over time. Then go from there. Usually memory leaks are fixed in newer versions.

[remz1337]

Great tuto! Wanted to add, there's now a script from tteck's proxmox helper script to automate Frigate LXC installation. See this discussion tteck/Proxmox#2711

[Redsandro]

This has been very helpful.

Since there has been a lot of bug reports with the 6.8 kernels, I would like to add that it is working with Proxmox 8.2 kernel 6.8.8-2-pve, at least with the Frigate 0.14 beta. It also works with the Ubuntu 24.04 image.

In both 22.04 and 24.04 though in order to use the iGPU for inferencing, you will need to install the Intel Compute Runtime files into the Frigate container after bringing it up:

docker exec -ti frigate bash
cd $(mktemp -d)

wget https://github.com/intel/intel-graphics-compiler/releases/download/igc-1.0.16900.23/intel-igc-core_1.0.16900.23_amd64.deb
wget https://github.com/intel/intel-graphics-compiler/releases/download/igc-1.0.16900.23/intel-igc-opencl_1.0.16900.23_amd64.deb
wget https://github.com/intel/compute-runtime/releases/download/24.22.29735.20/intel-level-zero-gpu_1.3.29735.20_amd64.deb
wget https://github.com/intel/compute-runtime/releases/download/24.22.29735.20/intel-opencl-icd_24.22.29735.20_amd64.deb
wget https://github.com/intel/compute-runtime/releases/download/24.22.29735.20/libigdgmm12_22.3.19_amd64.deb

dpkg -i *.deb

Some warnings will show, but they can be ignored. It will work, somehow. Keep in mind that the latest Proxmox, the latest Ubuntu and the latest Frigate are all changing rapidly so by the time you read this, it may not be necessary anymore. Try without first. For now it seems to be an outdated dependency: #12260

This is on slightly different hardware (Intel Core Ultra
with Intel Arc Alchemist iGPU) so YMMV.

I have tried doing idmaps and mapping the video and render group to the groups in the LXC but it just would NOT work

@Dougiebabe the format for setting this in the LXC configuration is new and confusing to me, but ChatGPT v4o explained it to me, and helped me create a configuration that works for me. Just paste in one of the examples, and explain to ChatGPT which ID's you need to map instead.

Use these commands inside the LXC container to find out where to map from, and again outsude the LXC container (i.e. on the Proxmox host) to find out where to map to.

getent group render
getent group video

In my case it would end up like this. /etc/pve/lxc/${LXCID}.conf:

lxc.idmap: u 0 100000 65535
lxc.idmap: g 0 100000 44
lxc.idmap: g 44 44 1
lxc.idmap: g 45 100045 62
lxc.idmap: g 108 104 1
lxc.idmap: g 109 100109 65427

It's a bit weird but you need to map all 65535 IDs in order, that's why you need all those extra lines that seem illogical at first, in stead of just the two you recognize.

hopefully someone can tell me if the chmod command below is OK?

The idmap is recommended because when your GPU is readable to all, it means everyone, even guests at any level inside or outside any container on the system, including unwanted guests from over the world, can potentially spy on your screen without having to do some kind of privilege escalation exploit. ⚠️

I realize I'm responding to an old post, but this is still very relevant.

[tdashmike]

@Redsandro Maybe you can help me out? Thank you!

#12328

[tdashmike]

I would rather not chmod renderD128 but I cannot get it to work with idmaps either. Any help is appreciated!

#12328

[tdashmike]

I tried chmod 666 today and it worked but I would still like to get idmap working. Thank you.

[HimbeersaftLP]

Maybe I am making a mistake there, but couldn't you just add it as a device passthrough from the Proxmox GUI and set the correct GID there?

[tdashmike]

I didn’t try that but I followed electricjacobs comment to run the docker container as user 0:106 which is the render group and it worked. Not exactly what I wanted because it was writing everything as render.

I am not using any acceleration now though. I don’t know why but my host would freeze with iGPU hardware acceleration.

https://www.reddit.com/r/selfhosted/comments/1agjixm/immich_hardware_acceleration_in_an_lxc_on_proxmox/

[AussiSG]

Maybe I am making a mistake there, but couldn't you just add it as a device passthrough from the Proxmox GUI and set the correct GID there?

I think this will work indeed.
I haven't tried this.

But my post from a few months ago almost shows the same

#5773 (comment)