How to convert depth maps obtained by orthogonal cameras into 3D point cloud

[flamehaze1115]

Describe the issue

Hello. Thanks for the great work.
I am using BlenderProc, with orthogonal cameras to render color images and depth maps, when I would like to convert the depth maps into 3D, I met some problem.
I know orthogonal camera is unlike pinhole camera, I use that code to do the conversion:

When I fuse two depth maps using cam2world matrices, the fused point cloud can be roughly matched but they are not aligned well.
I don't know what the reason is? maybe my conversion code is wrong?

Minimal code example

No response

Files required to run the code

No response

Expected behavior

Hoping for your help.

BlenderProc version

main branch

[flamehaze1115]

When I use ortho_scale=1, the pc from the two depth maps will be like this :

[flamehaze1115]

I get the problem. Blender ortho scale correponds a unit cube. But my code maps a (-1, 1) cube, should be "q = q / ortho_scale * 0.5"