A Pytorch implementation of Instant-NSR, fast surface reconstructor as described in Human Performance Modeling and Rendering via Neural Animated Mesh.
Based on dense multi-view input, our approach enables efficient and high-quality reconstruction, compression, and rendering of human performances. It supports 4D photo-real content playback for various immersive experiences of human performances in virtual and augmented reality.Human Performance Modeling and Rendering via Neural Animated Mesh
ACM Transactions on Graphics (SIGGRAPH Asia 2022)
Project page / Paper / Video
This repo helps us to reconstruct 3D models from multi-view images in ~10 mins. Free to run our code!
First, you need to set training
pip install -r requirements.txt
# (optional) install the tcnn backbone
pip install git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch
Tested on Ubuntu with torch 1.10 & CUDA 11.4 on RTX 3090.
We use the same data format as nerf and instant-ngp, and we provide a test dataset dance which is on google driver.
Please download and put it under {INPUTS}/dance
and then run our Instant-NSR code.
First time running will take some time to compile the CUDA extensions.
Train your own models, you can run following shell:
# Instant-NSR Training
CUDA_VISIBLE_DEVICES=${CUDA_DEVICE} python train_nerf.py "${INPUTS}/dance" --workspace "${WORKSPACE}" --downscale 1 --network sdf
Then, you can extract surface from the trained network model by:
# Instant-NSR Mesh extraction
CUDA_VISIBLE_DEVICES=${CUDA_DEVICE} python train_nerf.py "${INPUTS}/dance" --workspace "${WORKSPACE}" --downscale 1 --network sdf -mode mesh
Or, you can render target view with spefic camera view:
# Instant-NSR Rendering
CUDA_VISIBLE_DEVICES=${CUDA_DEVICE} python train_nerf.py "${INPUTS}/dance" --workspace "${WORKSPACE}" --downscale 1 --network sdf -mode render
Here are some reconstruction results from our Instant-NSR code:
Our code is implemented on torch-ngp code base:
@misc{torch-ngp,
Author = {Jiaxiang Tang},
Year = {2022},
Note = {https://github.com/ashawkey/torch-ngp},
Title = {Torch-ngp: a PyTorch implementation of instant-ngp}
}