-
Notifications
You must be signed in to change notification settings - Fork 28
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.
Already on GitHub? Sign in to your account
Slow TensorRT Inference Speed on Jetson Orin NX #35
Comments
Previously I have never seen tensorrt runs slower than onnx. |
Thanks for your prompt reply! Am I correct in understanding that if nothing goes wrong during the conversion of onnx files to trt files, the acceleration should theoretically be achieved? |
Yes. will you try tensorrt cpp version? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Thank you for your excellent work! 馃槅 馃槅 馃槅
Recently, I have been trying to use TensorRT to accelerate Depth Anything on Jetson Orin NX. However, I found that the inference speed of the converted TRT file does not significantly improve compared to the ONNX file, and it even decreases. Specifically:
The library versions are as follows:
The function to convert the .pth file to an ONNX file is as follows:
The function to convert the ONNX file to a TRT file is as follows:
The function to perform inference using the TRT file is as follows:
The code runs without any issues, except for some warnings during the ONNX conversion. However, the final results are still not satisfactory. Looking forward to your response! 鉂わ笍 鉂わ笍 鉂わ笍
The text was updated successfully, but these errors were encountered: