Preprocessing Frames and Inference on L40 GPU(runpod.io) is taking too much time.

[tariksetia]

Consider the following note book: https://github.com/tariksetia/cv-benchmark/blob/main/03-grounding-dino-gh.ipynb

If I run this on MacBook or v100 (pagemaker), the preprocessing of 283 images, take no time. But on L40 GPU(runpod) it takes 7 minutes. Don't know why?

The same difference is observed in model inference...ie one V100 during inference of 283 images with batch size of one , GPU utilisation is quite high reaching upto 100%. This leads to fps of 5 on 283 images.

But on L40, GPU utilisation never went more than 30. Most of the time it remained 1 %

The same notebook was run everywhere. and same behaviour is observed for GDino on huggingface.

Has anyone observed this earlier? Am I doing something wrong here?