The Stable diffusion code is copied from https://github.com/mindspore-lab/minddiffusion
pip install mindspore==1.9.0
pip install -r requirements.txt
-
download pytorch controlnet checkpoints from https://huggingface.co/lllyasviel/ControlNet/tree/main/models or https://huggingface.co/lllyasviel/ControlNet-v1-1/tree/main.
-
convert downloaded pytorch checkpoints to mindspore checkpoints, or directly download from https://huggingface.co/unrealMJ/MindSpore-ControlNet and put them into torch2ms/ms_weight
python torch2ms/convert.py --input_path xxxx --output_path xxxx # convert full model python torch2ms/convert.py --input_path xxxx --output_path xxxx --only_controlnet # convert controlnet only
-
Run run_controlnet_inference.py to use controlnet.
python run_controlnet_inference.py --input_path xxxx --output_path xxxx
-
Download the dataset from https://huggingface.co/datasets/fusing/fill50k
-
Run run_controlnet_train.py to train controlnet.
python run_controlnet_train.py --data_path xxxx --train_config configs/train_controlnet_config.json --model_config configs/cldm_v15.yaml