This repository contains an OpenGL pipeline, to render object RGBA-D layers from real 3D reconstructions (Stanford 2D-3D S). We provide C++ rendering code together with Python post-processing scripts. These layers can be merged based on per-pixel depth order to create Layered Depth Images (LDI). If you find this code useful, consider citing
@inproceedings{dhamo2019,
title={Object-Driven Multi-Layer Scene Decomposition From a Single Image},
author={Dhamo, Helisa and Navab, Nassir and Tombari, Federico},
booktitle={ICCV},
year={2019}
We used this rendered data to train the learned models from this paper.
- CMake 2.8 or above
- OpenCV 3.0 or above
- OpenGL 3.3
- Assimp
We tested the code with cuda=9.0, but other versions should be possible.
Download the Stanford 2D-3D S dataset from the official site .
Clone this repository and move to the root directory
git clone https://github.com/he-dhamo/OMLD-rendering.git
cd OMLD-rendering/
Prepare files to index pose and rgb files of the stanford data, for every area:
chmod +x prepare_filelists.sh
./prepare_filelists.sh <stanford_root_path>
Create output directories for every area
chmod +x make_dirs.sh
./make_dirs.sh
The default output directory is set to ./generated_images
. If you change this, please update the config file with the same output dir.
Create build directory and move there: mkdir build && cd build
Set options and paths in include/config.h
Run CMake: cmake ..
Compile: make
Run executable: ./RenderLayers
The program will save output images under OUT_DIR
, in a depth/
, rgb/
and instance/
folder; e.g. depth/<frame_idx>.png
for the whole image as well as depth/<frame_idx>_<layer_idx>.png
for each instance layer. In addition, we save the camera intrinsics and original frame file name (for all generated frames) in metadata.json
.
The C++ code is based on Learn OpenGL.
The post_pocessing/
directory contains python scripts that:
merge_layout.py
merge structural components (wall, floor, window, ceiling) to a common layout layer. The script will remove the current layers and create new ones including a layout layer.
python merge_layout.py --area area_1 --global_dir ../generated_images/
create_splits.py
create train and validation splits. The layers for training and testing are selected based on the level of amodal overlap with other layers. For example, to create the train splits:
python create_splits.py --split train --resume_json_state no --global_dir ../generated_images/