Official Pytorch implementation of DITTO: Dual and Integrated Latent Topologies for Implicit 3D Reconstruction (CVPR 2024)
@misc{shim2024ditto,
title={DITTO: Dual and Integrated Latent Topologies for Implicit 3D Reconstruction},
author={Jaehyeok Shim and Kyungdon Joo},
year={2024},
eprint={2403.05005},
archivePrefix={arXiv},
primaryClass={cs.CV}
}Please consider click the ⭐STAR⭐ if you think our project helpful!!
Please install the following packages as your own.
Then, install python packages:
pip install h5py einops scipy pykdtree open3d==0.18.0 kitsu==0.1.2Then, compile cython package:
python setup.py build_ext --inplaceWe use ShappeNet and SyntheticRooms datasets for training.
You can download the datasets from ConvONet's github page.
Please download datasets and unzip them so that have directory structures like following:
+ DITTO_CVPR24
+ assets
+ src
+ main.py
+ data
+ occupancynetwork
+ ShapeNet
+ 02691156
...
+ metadata.yaml
+ synthetic_room_dataset
+ rooms_04
...
+ rooms_08
Or you can modify *.yaml config files to designate your own directory path.
We use Scannet only for evaluation only with trained models in SyntheticRooms dataset following the convention.
Please follow preprocessing preprocess of ALTO's script, then move the preprocessed dataset folders following:
+ DITTO_CVPR24
+ assets
+ src
+ main.py
+ data
+ occupancynetwork
+ synthetic_room_dataset
+ scannet
+ scannet_v2 (actually it is not required, just for example)
+ scans
...
+ scannet_v2_alto (this is the target folder generated by preprocessing script)
+ scans
+ scenes
...
+ scenes
- SensorData.py
Or you can modify [src/generate_scannet.py](src/generate_scannet.py) file to designate your own directory path.
You can download every pretrained checkpoint in here.
wget https://github.com/vision3d-lab/ditto/releases/download/checkpoint/ditto_checkpoints.zip
unzip ditto_checkpoints.zip -d resultsThen, the checkpoints will be located like:
+ results
+ ditto
+ shapenet
+ 240301_0000_00
- args.yaml
- best.pth
+ 240301_0000_01.pts1000
- args.yaml
- best.pth
+ 240301_0000_02.pts300
...
+ synthetic_rooms_triplane
+ 240301_0000_00
...
+ 240301_0000_01.pts3000
...
+ 240301_0000_02.noisy
...
+ synthetic_rooms_voxel
+ 240301_0000_00
...
+ 240301_0000_01.sparse
...
You can find ShapeNet reconstruction demo using DITTO pretrained model in this notebook.
The training command has same structure regardless on the model and dataset:
python main.py {yaml config file path} --gpus {gpu ids separated by ','}The yaml config file possess information required to instantiate model, optimizer, dataset, trainer, e.t.c.
Here are some examples of training commands:
python main.py src/ditto/config/shapenet/00.yaml --gpus 0
python main.py src/ditto/config/shapenet/01.pts1000.yaml --gpus 0,1,2,3,4,5,6,7
python main.py src/ditto/config/synthetic_rooms_triplane/01.pts3000.yaml --gpus 0,1,2,3These commands will create directory named ./results/ditto/shapenet/{yaml_file_name}, and store logs and checkpoints inside of the directory.
Please replace {} part into the GPU numbers.
# 3K points & 0.005 noise
python main.py src/ditto/config/shapenet/00.yaml --gpus {}
# 1K points & 0.005 noise
python main.py src/ditto/config/shapenet/01.pts1000.yaml --gpus {}
# 300 points & 0.005 noise
python main.py src/ditto/config/shapenet/02.pts300yaml --gpus {}Please replace {} part into the GPU numbers.
# Triplane, 10K points & 0.005 noise
python main.py src/ditto/config/synthetic_rooms_triplane/00.yaml --gpus {}
# Triplane, 3K points & 0.005 noise
python main.py src/ditto/config/synthetic_rooms_triplane/01.pts3000.yaml --gpus {}
# Triplane, 10K points & 0.025 noise
python main.py src/ditto/config/synthetic_rooms_triplane/02.noisy.yaml --gpus {}
# Voxel, 10K points & 0.005 noise
python main.py src/ditto/config/synthetic_rooms_voxel/00.yaml --gpus {}
# Voxel, 3K points & 0.005 noise
python main.py src/ditto/config/synthetic_rooms_voxel/01.pts3000.yaml --gpus {}Generate results using src/ditto/generate.py and evaluate the generated meshes using src/ditto/eval.py:
PYTHONPATH=`pwd` python src/ditto/generate.py --resolution {128 for object, 256 for scene} {the_generated_experiments_directory_path}/args.yaml
PYTHONPATH=`pwd` python src/ditto/eval.py {the_generated_experiments_directory_path}/args.yamlHere are a few examples:
# ShapeNet
PYTHONPATH=`pwd` python src/ditto/generate.py --resolution 128 results/ditto/shapenet/240301_0000_00/args.yaml
PYTHONPATH=`pwd` python src/ditto/eval.py results/ditto/shapenet/240301_0000_00/args.yaml
# SyntheticRooms
PYTHONPATH=`pwd` python src/ditto/generate.py --resolution 256 results/ditto/synthetic_rooms_triplane/240301_0000_00/args.yaml
PYTHONPATH=`pwd` python src/ditto/eval.py results/ditto/synthetic_rooms_triplane/240301_0000_00/args.yamlPlease consider in mind to refer following works too.