Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switched to VideoHelper Load Audio (Error) Missing a file?? #8

Open
rethink-studios opened this issue Jun 19, 2024 · 6 comments
Open

Comments

@rethink-studios
Copy link

got prompt
'🔥 - 11 Nodes not included in prompt but is activated'
codeformer 24.0 video/h265-mp4
video h265-mp4
{'enhancer': 'codeformer', 'frame_enhancer': 'real_esrgan_x2', 'face_mask_padding_left': 0, 'face_mask_padding_right': 0, 'face_mask_padding_bottom': 0, 'face_mask_padding_top': 0, 'trim_frame_start': 0, 'trim_frame_end': 0, 'device': 'cpu', 'frame_rate': 24.0, 'loop_count': 0, 'filename_prefix': 'deepfuze', 'pingpong': False, 'save_output': True, 'images': ['2', 0], 'audio': ['22', 0], 'format': 'video/h265-mp4', 'pix_fmt': 'yuv420p10le', 'crf': 22, 'save_metadata': ['save_metadata', 'BOOLEAN', {'default': True}]}
D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-VideoHelperSuite\videohelpersuite..\video_formats\h265-mp4.json
pix_fmt
crf
save_metadata
Output images were not of valid resolution and have had padding applied
D:\COMFYUI_BETA\ComfyUI\input\audio\17188337500601833.wav
['python', './run.py', '--frame-processors', 'lip_syncer', '-s', 'D:\COMFYUI_BETA\ComfyUI\input\audio\17188337500601833.wav', '-t', 'D:\COMFYUI_BETA\ComfyUI\output\deepfuze_00011.mp4', '-o', 'D:\COMFYUI_BETA\ComfyUI\output\deepfuze\1718833750069751.mp4', '--trim-frame-start', '0', '--trim-frame-end', '310', '--face-mask-padding', '0', '0', '0', '0', '--headless']
Traceback (most recent call last):
File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-DeepFuze\run.py", line 7, in
from deepfuze import core
File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-DeepFuze.\deepfuze\core.py", line 16, in
from deepfuze.face_analyser import get_one_face, get_average_face
File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-DeepFuze.\deepfuze\face_analyser.py", line 10, in
from deepfuze.face_helper import estimate_matrix_by_face_landmark_5, warp_face_by_face_landmark_5, warp_face_by_translation, create_static_anchors, distance_to_face_landmark_5, distance_to_bounding_box, convert_face_landmark_68_to_5, apply_nms, categorize_age, categorize_gender
File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-DeepFuze.\deepfuze\face_helper.py", line 2, in
from cv2.typing import Size
ModuleNotFoundError: No module named 'cv2.typing'
['python', './run.py', '--frame-processors', 'face_enhancer', '--face-enhancer-model', 'codeformer', '-t', 'D:\COMFYUI_BETA\ComfyUI\output\deepfuze\1718833750069751.mp4', '-o', 'D:\COMFYUI_BETA\ComfyUI\output\deepfuze\enhanced_1718833750069751.mp4', '--headless']
Traceback (most recent call last):
File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-DeepFuze\run.py", line 7, in
from deepfuze import core
File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-DeepFuze.\deepfuze\core.py", line 16, in
from deepfuze.face_analyser import get_one_face, get_average_face
File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-DeepFuze.\deepfuze\face_analyser.py", line 10, in
from deepfuze.face_helper import estimate_matrix_by_face_landmark_5, warp_face_by_face_landmark_5, warp_face_by_translation, create_static_anchors, distance_to_face_landmark_5, distance_to_bounding_box, convert_face_landmark_68_to_5, apply_nms, categorize_age, categorize_gender
File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-DeepFuze.\deepfuze\face_helper.py", line 2, in
from cv2.typing import Size
ModuleNotFoundError: No module named 'cv2.typing'
['python', './run.py', '--frame-processors', 'frame_enhancer', '--frame-enhancer-model', 'real_esrgan_x2', '-t', 'D:\COMFYUI_BETA\ComfyUI\output\deepfuze\enhanced_1718833750069751.mp4', '-o', 'D:\COMFYUI_BETA\ComfyUI\output\deepfuze\enhanced_1718833750069751.mp4', '--headless']
Traceback (most recent call last):
File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-DeepFuze\run.py", line 7, in
from deepfuze import core
File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-DeepFuze.\deepfuze\core.py", line 16, in
from deepfuze.face_analyser import get_one_face, get_average_face
File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-DeepFuze.\deepfuze\face_analyser.py", line 10, in
from deepfuze.face_helper import estimate_matrix_by_face_landmark_5, warp_face_by_face_landmark_5, warp_face_by_translation, create_static_anchors, distance_to_face_landmark_5, distance_to_bounding_box, convert_face_landmark_68_to_5, apply_nms, categorize_age, categorize_gender
File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-DeepFuze.\deepfuze\face_helper.py", line 2, in
from cv2.typing import Size
ModuleNotFoundError: No module named 'cv2.typing'
ffmpeg version 2024-02-15-git-a2cfd6062c-essentials_build-www.gyan.dev Copyright (c) 2000-2024 the FFmpeg developers
built with gcc 12.2.0 (Rev10, Built by MSYS2 project)
configuration: --enable-gpl --enable-version3 --enable-static --pkg-config=pkgconf --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-zlib --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-sdl2 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-dxva2 --enable-d3d11va --enable-libvpl --enable-libgme --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libtheora --enable-libvo-amrwbenc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-librubberband
libavutil 58. 38.100 / 58. 38.100
libavcodec 60. 39.100 / 60. 39.100
libavformat 60. 21.100 / 60. 21.100
libavdevice 60. 4.100 / 60. 4.100
libavfilter 9. 17.100 / 9. 17.100
libswscale 7. 6.100 / 7. 6.100
libswresample 4. 13.100 / 4. 13.100
libpostproc 57. 4.100 / 57. 4.100
[in#0 @ 000001e0158e0e40] Error opening input: No such file or directory
Error opening input file D:\COMFYUI_BETA\ComfyUI\output\deepfuze\enhanced_1718833750069751.mp4.
Error opening input files: No such file or directory
None
None
!!! Exception during processing!!! _D:\COMFYUI_BETA\ComfyUI\output\deepfuze\enhanced_1718833750069751.mp4 could not be loaded with cv.
Traceback (most recent call last):
File "D:\COMFYUI_BETA\ComfyUI\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\COMFYUI_BETA\ComfyUI\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\COMFYUI_BETA\ComfyUI\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-DeepFuze\nodes.py", line 1425, in lipsyncgenerate
return load_video_cv(filename,0,'Disabled',512,512,0,0,1)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-DeepFuze\nodes.py", line 543, in load_video_cv
(width, height, fps, duration, total_frames, target_frame_time) = next(gen)
^^^^^^^^^
File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-DeepFuze\nodes.py", line 462, in cv_frame_generator
raise ValueError(f"{video} could not be loaded with cv.")
ValueError: _D:\COMFYUI_BETA\ComfyUI\output\deepfuze\enhanced_1718833750069751.mp4 could not be loaded with cv.

Prompt executed in 4.59 seconds

image

@SamKhoze
Copy link
Owner

Please make sure to read the instructions before seeking assistance. VHS node is not compatible to load audio for deepfuze We suggeste.

@rethink-studios
Copy link
Author

rethink-studios commented Jun 19, 2024

I understand, and trust me, I've tried both nodes. :)
Neither works, and I get the same error.
As you can see in the image, the other node is in the canvas

@SamKhoze
Copy link
Owner

@rethink-studios If you try both perhaps it's not installed correctly, I am sorry if this gives you a hard time, I wish I could find a one-click solution like other nodes, Lipsync, Faceswapping, Voice cloning, each of them are very complex even outside of comfyui it requires understanding the errors and know how to fix it. Let's try one more time, restart your comfyui and go to your terminal window, if you see any place "Error" or "NoModuleFound" take a screenshot and share it here.

@rethink-studios
Copy link
Author

rethink-studios commented Jun 20, 2024

WE BELIEVE IN YOU!! :)

attached is the error log in Comfy..

searching for "error"
if meet No module named 'hallo' error,please restart comfyui

ImportError: cannot import name 'create_vae_diffusers_config' from 'diffusers.loaders.single_file_utils' (D:\COMFYUI_BETA\python_embeded\Lib\site-packages\diffusers\loaders\single_file_utils.py)

raise ImportError("Diffusers version too old. Please update to 0.26.0 minimum.")

ImportError: Diffusers version too old. Please update to 0.26.0 minimum.

and this:

No OpenGL_accelerate module loaded: No module named 'OpenGL_accelerate'

deepFUZE_error2.txt

@rethink-studios
Copy link
Author

When I run this:
image

I get this:

To see the GUI go to: http:https://127.0.0.1:8188
FETCH DATA from: D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-Manager\extension-node-map.json [DONE]
BilboX LUTs path set to: D:\COMFYUI_BETA\ComfyUI\custom_nodes\bilbox-comfyui\luts
got prompt
'🔥 - 4 Nodes not included in prompt but is activated'
D:\COMFYUI_BETA\python_embeded\python.exe D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-Hallo\Hallo\scripts\inference.py --config "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-Hallo\tmp.yaml" --source_image "D:\COMFYUI_BETA\ComfyUI\input\unnamed.png" --driving_audio "D:\COMFYUI_BETA\ComfyUI\input\CLAIRE_YOURE_RIGHT_SOUFFLE_v01.mp3" --output D:\COMFYUI_BETA\ComfyUI\output\hallo_1718860470560910200.mp4 --pose_weight 1.0 --face_weight 1.0 --lip_weight 1.0 --face_expand_ratio 1.2
D:\COMFYUI_BETA\python_embeded\Lib\site-packages\diffusers\models\transformers\transformer_2d.py:34: FutureWarning: Transformer2DModelOutput is deprecated and will be removed in version 1.0.0. Importing Transformer2DModelOutput from diffusers.models.transformer_2d is deprecated and this will be removed in a future version. Please use from diffusers.models.modeling_outputs import Transformer2DModelOutput, instead.
deprecate("Transformer2DModelOutput", "1.0.0", deprecation_message)
2024-06-20 00:14:34.0948621 [E:onnxruntime:Default, provider_bridge_ort.cc:1548 onnxruntime::TryGetProviderInfo_CUDA] D:\a_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1209 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "D:\COMFYUI_BETA\python_embeded\Lib\site-packages\onnxruntime\capi\onnxruntime_providers_cuda.dll"

*************** EP Error ***************
EP Error D:\a_work\1\s\onnxruntime\python\onnxruntime_pybind_state.cc:857 onnxruntime::python::CreateExecutionProviderInstance CUDA_PATH is set but CUDA wasnt able to be loaded. Please install the correct version of CUDA andcuDNN as mentioned in the GPU requirements page (https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements), make sure they're in the PATH, and that your GPU is supported.
when using ['CUDAExecutionProvider', 'CPUExecutionProvider']
Falling back to ['CUDAExecutionProvider', 'CPUExecutionProvider'] and retrying.


2024-06-20 00:14:34.1682719 [E:onnxruntime:Default, provider_bridge_ort.cc:1548 onnxruntime::TryGetProviderInfo_CUDA] D:\a_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1209 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "D:\COMFYUI_BETA\python_embeded\Lib\site-packages\onnxruntime\capi\onnxruntime_providers_cuda.dll"

Traceback (most recent call last):
File "D:\COMFYUI_BETA\python_embeded\Lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 419, in init
self._create_inference_session(providers, provider_options, disabled_optimizers)
File "D:\COMFYUI_BETA\python_embeded\Lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 483, in _create_inference_session
sess.initialize_session(providers, provider_options, disabled_optimizers)
RuntimeError: D:\a_work\1\s\onnxruntime\python\onnxruntime_pybind_state.cc:857 onnxruntime::python::CreateExecutionProviderInstance CUDA_PATH is set but CUDA wasnt able to be loaded. Please install the correct version of CUDA andcuDNN as mentioned in the GPU requirements page (https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements), make sure they're in the PATH, and that your GPU is supported.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-Hallo\Hallo\scripts\inference.py", line 372, in
inference_process(command_line_args)
File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-Hallo\Hallo\scripts\inference.py", line 156, in inference_process
with ImageProcessor(img_size, face_analysis_model_path) as image_processor:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-Hallo\Hallo\hallo\datasets\image_processor.py", line 97, in init
self.face_analysis = FaceAnalysis(
^^^^^^^^^^^^^
File "D:\COMFYUI_BETA\python_embeded\Lib\site-packages\insightface\app\face_analysis.py", line 31, in init
model = model_zoo.get_model(onnx_file, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\COMFYUI_BETA\python_embeded\Lib\site-packages\insightface\model_zoo\model_zoo.py", line 96, in get_model
model = router.get_model(providers=providers, provider_options=provider_options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\COMFYUI_BETA\python_embeded\Lib\site-packages\insightface\model_zoo\model_zoo.py", line 40, in get_model
session = PickableInferenceSession(self.onnx_file, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\COMFYUI_BETA\python_embeded\Lib\site-packages\insightface\model_zoo\model_zoo.py", line 25, in init
super().init(model_path, **kwargs)
File "D:\COMFYUI_BETA\python_embeded\Lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 432, in init
raise fallback_error from e
File "D:\COMFYUI_BETA\python_embeded\Lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 427, in init
self._create_inference_session(self._fallback_providers, None)
File "D:\COMFYUI_BETA\python_embeded\Lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 483, in _create_inference_session
sess.initialize_session(providers, provider_options, disabled_optimizers)
RuntimeError: D:\a_work\1\s\onnxruntime\python\onnxruntime_pybind_state.cc:857 onnxruntime::python::CreateExecutionProviderInstance CUDA_PATH is set but CUDA wasnt able to be loaded. Please install the correct version of CUDA andcuDNN as mentioned in the GPU requirements page (https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements), make sure they're in the PATH, and that your GPU is supported.

Prompt executed in 4.07 seconds

@SamKhoze
Copy link
Owner

Hello @rethink-studios,

I think you are getting error in different custom node "IC-Light-ComfyUI-Node", can you ask them regrading this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants