Generation of 3D strand-based hairstyles from text.
[Paper] [Project Page] [Video]
Clone the repository and install requirements:
git clone https://github.com/Vanessik/HAAR
cd HAAR
conda env create -f enivironment.yml
conda activate haarInitialize submodules of k-diffusion, NeuralHaircut, LAVIS, LLaVA.
git submodule update --init --recursiveDownload the pretrained strand prior model from NeuralHaircut, diffusion checkpoint of HAAR model and HAAR data:
bash scripts/download.shThe code was tested on an NVIDIA A100 40GB GPU, and the environment specified in environment.yml is intended for Linux.
The full repository structure is organized as follows:
|-- src
|-- utils
|-- openaimodel.py
|-- sampler.py
|-- upsampling.py
|-- upsampler.py
|-- utils.py
|-- datasets
|-- dataset.py
|-- aug_dataset.py
|-- data
|-- examples
|-- pretrained_models
|-- strand_prior
|-- haar_prior
|-- applications
|-- image2hairstyle.py
|-- imagic_interpolation.py
|-- text_interpolation.py
|-- preprocess_dataset
|-- configs
|-- scripts
|-- train.py
|-- infer.py
|-- environment.yml
- For inference use the following command:
bash ./scripts/infer.shor
python infer.py --exp_name infer_haar --conf ./configs/infer.json --ckpt_path ./pretrained_models/haar_prior/haar_diffusion.pth --cfg_scale {CFG_SCALE} --save_latent_textures --save_guiding_strands --save_upsampled_hairstyle --upsample_resolution 64 --n_samples {NUM_VARIATIONS} --hairstyle_description {HAISRTYLE_DESCRIPTION}python infer.py --exp_name infer_haar --conf ./configs/infer.json --ckpt_path ./pretrained_models/haar_prior/haar_diffusion.pth --cfg_scale 1.5 --save_latent_textures --save_guiding_strands --save_upsampled_hairstyle --upsample_resolution 64 --n_samples 10 --hairstyle_description "a woman with short straight hairstyle"For more information on parameters:
python infer.py --help- Interpolate hairstyle between two text prompts:
python ./applications/text_interpolation.py --conf ./configs/infer.json --hairstyle_1 'straight woman hairstyle' --hairstyle_2 'long wavy haircut' --save_guiding_strands --save_upsampled_hairstyle --save_latent_textures --upsample_resolution 128 --cfg_scale 1.5 --seed 32 --n_interpolation_states 5- Editing of hairstyle with text prompt:
python ./applications/imagic_interpolation.py --conf ./configs/infer.json --hairstyle_edit_prompt 'short hairstyle' --target_hairstyle_texture 'exp_texture.pt' --save_guiding_strands --save_upsampled_hairstyle --save_latent_textures --upsample_resolution 128 --cfg_scale 1.5 --seed 32 --n_interpolation_states 5- Create hairstyle using an input image:
From input image obtain description of hairstyle using LLaVA and then create strand-based hairstyle from description.
bash scripts/image2hairstyle.shMore information can be found in preprocess_dataset.
For training use the following command:
bash scripts/train.shThis code and model are available for scientific research purposes as defined in the LICENSE.txt file. By downloading and using the project you agree to the terms in the LICENSE.txt.
This work is based on the great projects:
-
NeuralHaircut for strand-based prior model;
-
LAVIS for calculation of text embeddings;
-
k-diffusion for diffusion network;
-
LLaVA for hairstyle annotation;
- NeuralHaircut reconstructs strand-based hairstyle using multi-view data or monocular video;
Cite as below if you find this repository is helpful to your project:
@InProceedings{Sklyarova_2024_CVPR,
author = {Sklyarova, Vanessa and Zakharov, Egor and Hilliges, Otmar and Black, Michael J. and Thies, Justus},
title = {Text-Conditioned Generative Model of 3D Strand-based Human Hairstyles},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2024},
pages = {4703-4712}
}
