Cuda bindings mismatch error using custom trained model #32
Comments
|
Hi @immaixq I am also trying to use a custom yolov8 model, but failed to do so. I also converted the |
|
I suspect that the issue is in the hardcoded number of classes If your custom model has different number of classes then it will break here I suggest that you visualize your ONNX model using netron, and note the input/output names and the input/output dimensions. |
|
Hi @immaixq, The problem you posted seems to originate from a parameter mismatch. We have parameters called The input/output binding names should be noted from the ONNX model as @mzahana said. The input tensor names can be kept the same. The output tensor name I believe should be It also appears that your model outputs a 3 tensors, instead of 2 as we expect. Having additional information about your network would likely be useful. |
|
Hi @mzahana, To respond to your issue separately, hopefully my first response is helpful for your first issue. You're also right that the number of classes expected by the decoder node is hard coded, which will be problematic later. Currently, the issue seems to originate from the TensorRT node, and not the decoder. |
kajananchinniahNV
added
bug
|
@kajananchinniahNV thanks for your response. My problem was solved after I changed the hard-coded number of classes according to my custom model. I suggest to make it a parameter. Thanks. |
|
Thanks for finding a fix to the issue! I'll remove the We'll be working on a patch to this issue soon. The solution you recommend sounds reasonable to me. @immaixq could you confirm that this also solves it for you? |
|
@kajananchinniahNV thank you. Actually, to make the fix more complete, the output dimension of the model could be a parameter. This way, the decoder can adapt to custom models with output tensor dimensions(not only the number of classes) that is different from the default one by Yolov8 [1, 84, 8400]. Thanks. |
|
@mzahana Thank you for your help and advice, I have gained some insights after looking into netron again. I did changed the number of classes in the script and I think the error is caused by the number of outputs my model gives, which consists of output0 and output1. I managed to fix the mismatch error after changing @kajananchinniahNV Thank you for your help, I have changed the parameters as mentioned above and managed to fix the mismatch error. Unfortunately another error occurred. The process has died without additional log on its cause. it'd be great if you could share some insights or advice on how to troubleshoot the cause of it. Really appreciate both of your help, thank you!
|
|
I'll be reopening this until we have a solution merged in the repository for tracking purposes. We'll close it once a fix is in place. |
|
Hello, i'm coming from another issue post, that has the same problem. I did check the neutron model structure, but it did not differ from the original yolov8s model, except from the output size that goes from [1,84,8400] to [1,5,8400]. For this reason I added a parameter into the yolov8_decoder_node.hpp and yolov8_decoder_node.cpp that can be modified from the launch file, in order to take in account the class number change. But still everything crashes after launch, the problem seems to be inside the tensor_rt node. while here it is the run of the tesor_rt_node without any encoder/decoder: |
|
Hi @TeamRoboTo, I believe the issue is because the In the quickstart, the command we provide is: The |
|
Yep, thanks you. Now it works :) |
|
Sorry,I have the same issue.I changed the code cloned from git(num_classes), and then I found that running ros2 launch according to quickstart didn't seem to compile my code, so my changes won't take effect,I want to know the correct operation,any suggestions? |
|
Hi @magic-Alex In our latest release because of our usage of debians now, it's harder to update the source code directly to apply this fix now. My recommendation for now would be to uninstall the debians for this package and instead compile the code from source (via If you run into issues doing this, please let us know! |
|
@kajananchinniahNV Thanks,it works! |
|
I believe I'm having the same issue. I'm trying to run a model with the following input/output dimensions: ros2 launch isaac_ros_yolov8 isaac_ros_yolov8_visualize.launch.py model_file_path:=/workspaces/isaac_ros-dev/models/12_1_23-87_8.onnx engine_file_path:=/workspaces/isaac_ros-dev/models/12_1_23-87_8.plan input_binding_names:=['images'] output_binding_names:=['output0'] network_image_width:=640 network_image_height:=640 force_engine_update:=False image_mean:=[0.0,0.0,0.0] image_stddev:=[1.0,1.0,1.0] input_image_width:=640 input_image_height:=640 confidence_threshold:=0.25 nms_threshold:=0.45Is there anything I'm missing here? |
|
Hi @eddieliao , Could you please provide the full error trace? There can be many different reasons that the process dies. |
|
I've managed to get it to work with an image, but if I input a video it will crash when the video ends.
Here's the full output, although it doesn't seem too useful? |
Description:
Encounter error using yolov8_tensor_rt.launch.py and isaac_ros_yolov8_visualize.launch.py. The scripts are run on jetson orin nano. The custom trained model is exported to onnx as suggested by documentation using ultralytics, I have also tried exporting using trtexec but still face the same error.
Thank you for your help.
The text was updated successfully, but these errors were encountered: