Nvidia Tesla P4 GPU stuck in P0 state

[obones]

Hello,

I'm just starting to explore frigate and it works well "out of the box" despite having high CPU usage by ffmpeg.
Luckily, I a Nvidia Tesla P4 GPU card in the system so I configured frigate to tell ffmpeg to use it like so:

ffmpeg:
  hwaccel_args: preset-nvidia-h264

It works quite well, CPU usage is down to almost nothing but I have another issue that leads to GPU shut down because of overheating.
nvidia-smi tells me that the card keeps running in P0 power state, drawing 25W and never moving back to P2 as shown in the frigate documentation:

+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 530.41.03              Driver Version: 530.41.03    CUDA Version: 12.1     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                  Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf            Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  Tesla P4                        Off| 00000000:01:00.0 Off |                    0 |
| N/A   46C    P0               24W /  75W|    330MiB /  7680MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|    0   N/A  N/A    811348      C   ffmpeg                                      328MiB |
+---------------------------------------------------------------------------------------+

Does any of you have any idea what could be causing this? I'm quite positive this is an issue with the GPU configuration rather than with frigate but I could not find any solution and hope someone may have hints for a solution.

[NickM-27]

I'd suggest trying an updated driver if one is available.

I also wouldn't necessarily take that part of the frigate documentation as what it should look like, different GPUs will behave differently.

[NickM-27]

You can also use nvidia smi on the host to set the p state

[hardwareadictos]

I have the same issue with a P4, latest grid drivers:, GPU is attatched via vgpu to another 3 vms and it only reaches P8 state when i stop Frigate container, using hwaccel_args: preset-nvidia-h264 also, happens with hwaccel_args: preset-nvidia-h265 too.

Would be nice to know which behaviour are other nvidia users experiencing.