depth

[caiyongqi]

Hi. In the DTU dataset, I found that there is a big difference between the depth map estimated using colomap (you provided) and the ground truth depth map, have you checked this?

scan1, view 0, 300x400:
ground truth depth map:

depth map estimated using colomap:

Could you check it? Thanks!!!

[liuyuan-pal]

Does this refer to the training set of NeuRay? As for the DTU training dataset, I adopt the processed dataset from PixelNeRF and run the colmap based on this processed version. I'm not sure what cause the difference. It may be caused by the normalization of scale and offset. Since COLMAP indeed will produce some incorrect depth in test time, it would be better to let the model receive these incorrect depth values in training.

[caiyongqi]

Does this refer to the training set of NeuRay? As for the DTU training dataset, I adopt the processed dataset from PixelNeRF and run the colmap based on this processed version. I'm not sure what cause the difference. It may be caused by the normalization of scale and offset. Since COLMAP indeed will produce some incorrect depth in test time, it would be better to let the model receive these incorrect depth values in training.

OK, Thanks! Could you update how to use colomap to get depth map (code or document) if you have time? I believe it will be useful.

[liuyuan-pal]

You may refer to the document of COLMAP. It already provides patch_match_stereo to estimate depth maps and you can read these depth maps from the provided python function like

NeuRay/dataset/database.py

Line 129 in 266af33

return read_array(f'{self.root_dir}/colmap_depth/{img_id}.jpg.geometric.bin')