A HiDiffusion node for ComfyUI

--增加adapter_style支持，SDXL需求的显存较大，虽然能跑CPU，但是不推荐，会爆显存，SD1.5测试没问题。
--修复controlnet菜单的加载列表的问题；
--Adding adapter_style support, SDXL requires a large amount of graphics memory. Although it can run on CPU, it is not recommended as it may cause graphics memory "explosion". SD1.5 testing is not a problem.
--Fix the issue with loading lists in the Controlnet menu;

1、ParlerTTS node:ComfyUI_ParlerTTS
2、Llama3_8B node:ComfyUI_Llama3_8B
3、HiDiffusion node：ComfyUI_HiDiffusion_Pro
4、ID_Animator node： ComfyUI_ID_Animator
5、StoryDiffusion node：ComfyUI_StoryDiffusion
6、Pops node：ComfyUI_Pops
7、stable-audio-open-1.0 node ：ComfyUI_StableAudio_Open
8、GLM4 node：ComfyUI_ChatGLM_API
9、CustomNet node：ComfyUI_CustomNet
10、Pipeline_Tool node :ComfyUI_Pipeline_Tool
11、Pic2Story node :ComfyUI_Pic2Story
12、PBR_Maker node:ComfyUI_PBR_Maker
13、ComfyUI_Streamv2v_Plus node:ComfyUI_Streamv2v_Plus
14、ComfyUI_MS_Diffusion node:ComfyUI_MS_Diffusion

-- 增加 manne加速Lora
-- 加入controlnet-tile-sdxl的支持，内置图片预处理，默认512尺寸，新增apply_window_attn 条件控制。
--修复节点连接逻辑，现在文生图模式，无需接入image，无controlnet也无需接入control_image
--支持SDXL-lighting\Hyper\LCM\DMD2\的加速Unet，建议适当提高步数；
--基于官方的更新，加入lora支持，需要填关键词；
--加入skip，去掉意义不大的其他参数；
--支持所有的SDXL controlnet模型（仅支持配置config文件的diffuser加载模式）；
--你可以修改model.yaml文件，添加其他的可能支持SDXL“扩散模型”或者“controlnet”或者SDXL“unet”模型；

-- add manne lighting lora
--Added support for control net file sdxl, built-in image preprocessing, default size of 512, and added condition control for apply_window_attn.
--Fix node connection logic, now in text-based graphics mode, there is no need to connect to image, no controllnet, and no need to connect to controll_image
--Support acceleration Unet for SDXL lighting, Hyper, LCM, and DMD2. It is recommended to increase the number of steps appropriately;
--Based on official updates, adding support for Lora requires filling in keywords;
--Add skip and remove other parameters that are not significant;
--Supports all SDXL controllnet models (only supports diffuser loading mode for configuring config files);
--You can modify the model.yaml file and add other models that may support SDXL "diffusion models" or "controllnet" or SDXL "unet" models;

1.1 In the .\ComfyUI \ custom_node directory, run the following:

git clone https://github.com/smthemex/ComfyUI_HiDiffusion_Pro.git   

1.2 using it

diffusers >=0.28.0
yaml

3种调用模型节点的方法
一种是:在repo_id填写诸如"stabilityai/stable-diffusion-xl-base-1.0" 这样的标准抱脸的repo_id,系统会自己下载或者查找.cache路径，这个要开全局外网或者你开启了hf镜像

一种是:把你下载好的模型放在comfyUI的"models/diffusers"路径下，记得不要改最尾的模型路径名“比如stable-diffusion-xl-base-1.0”，就可以用菜单直接调用。（注意：要用菜单的方式，必须删掉repo_id默认的"stabilityai/stable-diffusion-xl-base-1.0"，让repo_id留空，controlnet_id一样，要调用菜单，必须留空controlnet_id）

一种是:你在repo_id或controlnet_id直接填写诸如“x:/xx/xx/stabilityai/stable-diffusion-xl-base-1.0” 这样的已经下载好的扩散模型的绝对路径也能用

Three methods for calling model nodes
Filling in the corresponding repo_id will download the model
or
in the diffuse directory, store and use the downloaded model （Notice:in this method,repo_id or controlnet_repo_id must be empty）
or
using repo_id,and Fill in the absolute path of your diffusers model,using"/"

controlnet模型存放示例：

├── ComfyUI/models/diffusers/   
|     ├──xinsir/controlnet-openpose-sdxl-1.0    
|         ├── config.json   
|         ├── diffusion_pytorch_model.fp16.safetensors   
|     ├──xinsir/controlnet-scribble-sdxl-1.0   
|         ├── config.json   
|         ├── diffusion_pytorch_model.fp16.safetensors
|     ├──xinsir/controlnet-tile-sdxl-1.0  
|         ├── config.json   
|         ├── diffusion_pytorch_model.fp16.safetensors  
|     ├──diffusers/controlnet-canny-sdxl-1.0   
|         ├── config.json   
|         ├── diffusion_pytorch_model.fp16.safetensors   
|     ├──diffusers/controlnet-depth-sdxl-1.0   
|         ├── config.json   
|         ├── diffusion_pytorch_model.fp16.safetensors
|     ├──TheMistoAI/MistoLine 
|         ├── config.json   
|         ├── diffusion_pytorch_model.fp16.safetensors

ip_adapter model 模型存放示例：

如果不存放对应，运行时会自动下载。 If the corresponding file is not stored, it will be automatically downloaded at runtime.

├── ComfyUI/custom_nodes/ComfyUI_HiDiffusion_Pro/weights/  
|     ├──models
|         ├── ip-adapter_sd15.bin
|         ├── image_encoder/
|             ├── config.json
|             ├── model.safetensors
|     ├──sdxl_models
|         ├── ip-adapter_sdxl.bin
|         ├── image_encoder/
|             ├── config.json
|             ├── model.safetensors

部分支持的模型请查看model.yaml 文件
For partially supported models, please refer to the model.yaml file

txt2img using ip_adapter_style

img2img use controlnet and lora 图生图加controlnet和lora

use inpainting XL model 内绘

txt2img use XL/sd1.5/XL turbo/background... 文生图

txt2img + controlnet 文生图加controlnet .

img2img use Hyper unet 图生图加加速unet

@article{zhang2023hidiffusion,
  title={HiDiffusion: Unlocking Higher-Resolution Creativity and Efficiency in Pretrained Diffusion Models},
  author={Zhang, Shen and Chen, Zhaowei and Zhao, Zhenyu and Chen, Yuhao and Tang, Yao and Liang, Jiajun},
  journal={arXiv preprint arXiv:2311.17528},
  year={2023}
}