-
Notifications
You must be signed in to change notification settings - Fork 34
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Detection Evaluation Script in Python #8
Comments
Hi, Thanks for your feedback! However, since the evaluation kit is provided by Coop, which is implemented in Matlab, and there is no further plan to migrate it to Python, sorry that it can not be integrated into the testing process easily. You may try to reimplement a Python version of the detection evaluation, which might be not easy. |
Appreciate the response. |
Hi @chengzhag , So I was trying to get the mAP evaluation scores but it seems that the 'co_bdb_3d.mat' and 'co_layout.mat' files are not being generated in the test set. I dug into it a bit and it seems that models/total3d/dataloader.py does not load the 'world_R_inv' key in the 'camera' dict. Can you please advice me on how to work around/fix this? |
Hi, @metro-smiles, In order to evaluate mAP, you need to generate dataset with our modified data processing code as mentioned by README. |
Hi @chengzhag , |
Hi, @Ivan-VV, To be more specific, since Total3D has not provided evaluation code, we use code from Coop for evaluation. However, Total3D uses a coordinate frame different from Coop. So we implement a transformation code for different coordinate frames. However, the transformation is different for each scene and is generated while preparing the dataset with our modified code, and will be needed here when testing (if not exist, the testing process will not generate 'co_bdb_3d.mat'). Thus, you need to prepare the dataset with our modified data processing code in order to do the evaluation. |
Thank you for your reply! @chengzhag |
Hi @chengzhag ,
Great work indeed! I had one quick question. Is the evaluation script (for object detection) also available in Python, so that it could be integrated with the testing module directly?
Appreciate your help.
Cheers!
The text was updated successfully, but these errors were encountered: