question about the difference between the intrinsics and extrinsics calibrated and estimated by ROMP

[sycamore27]

Hello, I use ROMP to estimate the SMPL and use it to render novel view pictures. But I found that if using the intrinsics and extrinsics estimated by ROMP can correctly generate the bounding box in 2D picture, but using intrinsics and extrinsics calibrated by the camera ourselves will be wrong, which is really inexplicable. Why can't I use the intrinsics and extrinsics calibrated by the camera? Do you know the possible reason? Thank you very much!

[Arthur151]

Hi. You can use our estimated 2D keypoints and 3D keypoints to re-calcualte the 3D translation of people via PnP.
The reason of failure is that the 3D translation we estimated is just for our pre-defined camera, instead of any other pecific camera. It would be helpful to determine the bug is you can provide the details of your intrinsics and extrinsics.
The PnP could refer to

ROMP/simple_romp/romp/utils.py

Line 331 in bafc868

def estimate_translation_cv2(joints_3d, joints_2d, focal_length=600, img_size=np.array([512.,512.]), proj_mat=None, cam_dist=None):

You can replace the camMat with your intrinsics

[sycamore27]

Thank you for your reply! The correct 2D bounding box can be obtained by recalculating 3D translation with my intrinsics by the estimate_translation_cv2 function.

However, there is another question. How can I get the correct smpl and corresponding 2D bounding box when I have set an exact camera position, in other words, given an exact extrinsics? Thank you very much!

[Arthur151]

Sorry, this feature is not supported by the current version, but will be supported by the next version.
Best