You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
问题如下:
CUDA SETUP: Something unexpected happened. Please compile from source:
git clone [email protected]:TimDettmers/bitsandbytes.git
cd bitsandbytes
CUDA_VERSION=124
python setup.py install
CUDA SETUP: Setup Failed!
CUDA SETUP: Something unexpected happened. Please compile from source:
git clone [email protected]:TimDettmers/bitsandbytes.git
cd bitsandbytes
CUDA_VERSION=124
python setup.py install
╭───────────────────── Traceback (most recent call last) ──────────────────────╮
│ /home/ToolBench-master/toolbench/inference/qa_pipeline.py:13 in │
│ │
│ 10 os.environ['CUDA_VERSION'] = "12.4" │
│ 11 os.environ['NVCC'] = '/usr/local/cuda-12.4/bin/nvcc' │
│ 12 │
│ ❱ 13 from toolbench.inference.Downstream_tasks.rapidapi import pipeline_runn │
│ 14 │
│ 15 │
│ 16 if name == "main": │
│ │
│ /home/ToolBench-master/toolbench/inference/Downstream_tasks/rapidapi.py:1 │
│ 0 in │
│ │
│ 7 from termcolor import colored │
│ 8 import random │
│ 9 from toolbench.inference.LLM.chatgpt_function_model import ChatGPTFunc │
│ ❱ 10 from toolbench.inference.LLM.davinci_model import Davinci │
│ 11 from toolbench.inference.LLM.tool_llama_lora_model import ToolLLaMALoR │
│ 12 from toolbench.inference.LLM.tool_llama_model import ToolLLaMA │
│ 13 from toolbench.inference.LLM.retriever import ToolRetriever │
│ │
│ /home/ToolBench-master/toolbench/inference/LLM/davinci_model.py:9 in │
│ │
│ │
│ 6 import random │
│ 7 import openai │
│ 8 from typing import Optional │
│ ❱ 9 from toolbench.model.model_adapter import get_conversation_template │
│ 10 from toolbench.inference.utils import SimpleChatIO, react_parser │
│ 11 from toolbench.inference.Prompts.ReAct_prompts import FORMAT_INSTRUCTI │
│ 12 │
│ │
│ /home/ToolBench-master/toolbench/model/init.py:1 in │
│ │
│ ❱ 1 from toolbench.model.model_adapter import ( │
│ 2 │ load_model, │
│ 3 │ get_conversation_template, │
│ 4 │ add_model_args, │
│ │
│ /home/ToolBench-master/toolbench/model/model_adapter.py:21 in │
│ │
│ 18 │ AutoTokenizer, │
│ 19 │ LlamaForCausalLM, │
│ 20 ) │
│ ❱ 21 from peft import PeftModel │
│ 22 │
│ 23 from toolbench.tool_conversation import Conversation, get_conv_templat │
│ 24 from toolbench.model.compression import load_compress_model │
│ │
│ /home/anaconda3/envs/toolllm/lib/python3.11/site-packages/peft/init.py:2 │
│ 2 in │
│ │
│ 19 │
│ 20 version = "0.3.0" │
│ 21 │
│ ❱ 22 from .mapping import MODEL_TYPE_TO_PEFT_MODEL_MAPPING, PEFT_TYPE_TO_CON │
│ 23 from .peft_model import ( │
│ 24 │ PeftModel, │
│ 25 │ PeftModelForCausalLM, │
│ │
│ /home/anaconda3/envs/toolllm/lib/python3.11/site-packages/peft/mapping.py:16 │
│ in │
│ │
│ 13 # See the License for the specific language governing permissions and │
│ 14 # limitations under the License. │
│ 15 │
│ ❱ 16 from .peft_model import ( │
│ 17 │ PeftModel, │
│ 18 │ PeftModelForCausalLM, │
│ 19 │ PeftModelForSeq2SeqLM, │
│ │
│ /home/anaconda3/envs/toolllm/lib/python3.11/site-packages/peft/peft_model.py │
│ :31 in │
│ │
│ 28 from transformers.modeling_outputs import SequenceClassifierOutput, T │
│ 29 from transformers.utils import PushToHubMixin │
│ 30 │
│ ❱ 31 from .tuners import ( │
│ 32 │ AdaLoraModel, │
│ 33 │ AdaptionPromptModel, │
│ 34 │ LoraModel, │
│ │
│ /home/anaconda3/envs/toolllm/lib/python3.11/site-packages/peft/tuners/__init │
│ __.py:21 in │
│ │
│ 18 # limitations under the License. │
│ 19 │
│ 20 from .adaption_prompt import AdaptionPromptConfig, AdaptionPromptModel │
│ ❱ 21 from .lora import LoraConfig, LoraModel │
│ 22 from .adalora import AdaLoraConfig, AdaLoraModel │
│ 23 from .p_tuning import PromptEncoder, PromptEncoderConfig, PromptEncoder │
│ 24 from .prefix_tuning import PrefixEncoder, PrefixTuningConfig │
│ │
│ /home/anaconda3/envs/toolllm/lib/python3.11/site-packages/peft/tuners/lora.p │
│ y:40 in │
│ │
│ 37 │
│ 38 │
│ 39 if is_bnb_available(): │
│ ❱ 40 │ import bitsandbytes as bnb │
│ 41 │
│ 42 │
│ 43 @DataClass │
│ │
│ /home/anaconda3/envs/toolllm/lib/python3.11/site-packages/bitsandbytes-0.38. │
│ 0.post2-py3.11.egg/bitsandbytes/init.py:7 in │
│ │
│ 4 # LICENSE file in the root directory of this source tree. │
│ 5 │
│ 6 from . import cuda_setup, utils │
│ ❱ 7 from .autograd._functions import ( │
│ 8 │ MatmulLtState, │
│ 9 │ bmm_cublas, │
│ 10 │ matmul, │
│ │
│ /home/anaconda3/envs/toolllm/lib/python3.11/site-packages/bitsandbytes-0.38. │
│ 0.post2-py3.11.egg/bitsandbytes/autograd/init.py:1 in │
│ │
│ ❱ 1 from ._functions import undo_layout, get_inverse_transform_indices │
│ 2 │
│ │
│ /home/anaconda3/envs/toolllm/lib/python3.11/site-packages/bitsandbytes-0.38. │
│ 0.post2-py3.11.egg/bitsandbytes/autograd/_functions.py:9 in │
│ │
│ 6 │
│ 7 import torch │
│ 8 │
│ ❱ 9 import bitsandbytes.functional as F │
│ 10 │
│ 11 │
│ 12 # math.prod not compatible with python < 3.8 │
│ │
│ /home/anaconda3/envs/toolllm/lib/python3.11/site-packages/bitsandbytes-0.38. │
│ 0.post2-py3.11.egg/bitsandbytes/functional.py:17 in │
│ │
│ 14 from typing import Tuple │
│ 15 from torch import Tensor │
│ 16 │
│ ❱ 17 from .cextension import COMPILED_WITH_CUDA, lib │
│ 18 │
│ 19 │
│ 20 # math.prod not compatible with python < 3.8 │
│ │
│ /home/anaconda3/envs/toolllm/lib/python3.11/site-packages/bitsandbytes-0.38. │
│ 0.post2-py3.11.egg/bitsandbytes/cextension.py:20 in │
│ │
│ 17 │ if lib is None and torch.cuda.is_available(): │
│ 18 │ │ CUDASetup.get_instance().generate_instructions() │
│ 19 │ │ CUDASetup.get_instance().print_log_stack() │
│ ❱ 20 │ │ raise RuntimeError(''' │
│ 21 │ │ CUDA Setup failed despite GPU being available. Please run the f │
│ 22 │ │ │
│ 23 │ │ python -m bitsandbytes │
╰──────────────────────────────────────────────────────────────────────────────╯
RuntimeError:
CUDA Setup failed despite GPU being available. Please run the following
command to get more information:
python -m bitsandbytes
Inspect the output of the command and see if you can locate CUDA
libraries. You might need to add them
to your LD_LIBRARY_PATH. If you suspect a bug, please take the
information from python -m bitsandbytes
and open an issue at: https://github.com/TimDettmers/bitsandbytes/issues
./scripts/inference_toolllama_helloworld_demo.sh:行3: --tool_root_dir: 未找到命令
./scripts/inference_toolllama_helloworld_demo.sh:行4: --backbone_model: 未找到命令
./scripts/inference_toolllama_helloworld_demo.sh:行5: --model_path: 未找到命令
./scripts/inference_toolllama_helloworld_demo.sh:行6: --max_observation_length: 未找到命令
./scripts/inference_toolllama_helloworld_demo.sh:行7: --observ_compress_method: 未找到命令
./scripts/inference_toolllama_helloworld_demo.sh:行8: --method: 未找到命令
./scripts/inference_toolllama_helloworld_demo.sh:行9: --input_query_file: 未找到命令
./scripts/inference_toolllama_helloworld_demo.sh:行10: --output_answer_file: 未找到命令
./scripts/inference_toolllama_helloworld_demo.sh:行11: --api_customization: 未找到命令
终端输入nvcc -V
能看出来cuda是正确安装的,
找不到解决办法了55555555,特来问问大家,有知道怎么解决的嘛
The text was updated successfully, but these errors were encountered:
配置:
系统:CentOS Linux release 7.9.2009
CUDA版本: 12.4
python版本: 3.11
NVIDIA-SMI :550.54.14
我在使用hello_world推理时遇到了这个问题,已经提示重新编译了bitsandbytes,但还是出现了这个问题。
问题如下:
CUDA SETUP: Something unexpected happened. Please compile from source:
git clone [email protected]:TimDettmers/bitsandbytes.git
cd bitsandbytes
CUDA_VERSION=124
python setup.py install
CUDA SETUP: Setup Failed!
CUDA SETUP: Something unexpected happened. Please compile from source:
git clone [email protected]:TimDettmers/bitsandbytes.git
cd bitsandbytes
CUDA_VERSION=124
python setup.py install
╭───────────────────── Traceback (most recent call last) ──────────────────────╮
│ /home/ToolBench-master/toolbench/inference/qa_pipeline.py:13 in │
│ │
│ 10 os.environ['CUDA_VERSION'] = "12.4" │
│ 11 os.environ['NVCC'] = '/usr/local/cuda-12.4/bin/nvcc' │
│ 12 │
│ ❱ 13 from toolbench.inference.Downstream_tasks.rapidapi import pipeline_runn │
│ 14 │
│ 15 │
│ 16 if name == "main": │
│ │
│ /home/ToolBench-master/toolbench/inference/Downstream_tasks/rapidapi.py:1 │
│ 0 in │
│ │
│ 7 from termcolor import colored │
│ 8 import random │
│ 9 from toolbench.inference.LLM.chatgpt_function_model import ChatGPTFunc │
│ ❱ 10 from toolbench.inference.LLM.davinci_model import Davinci │
│ 11 from toolbench.inference.LLM.tool_llama_lora_model import ToolLLaMALoR │
│ 12 from toolbench.inference.LLM.tool_llama_model import ToolLLaMA │
│ 13 from toolbench.inference.LLM.retriever import ToolRetriever │
│ │
│ /home/ToolBench-master/toolbench/inference/LLM/davinci_model.py:9 in │
│ │
│ │
│ 6 import random │
│ 7 import openai │
│ 8 from typing import Optional │
│ ❱ 9 from toolbench.model.model_adapter import get_conversation_template │
│ 10 from toolbench.inference.utils import SimpleChatIO, react_parser │
│ 11 from toolbench.inference.Prompts.ReAct_prompts import FORMAT_INSTRUCTI │
│ 12 │
│ │
│ /home/ToolBench-master/toolbench/model/init.py:1 in │
│ │
│ ❱ 1 from toolbench.model.model_adapter import ( │
│ 2 │ load_model, │
│ 3 │ get_conversation_template, │
│ 4 │ add_model_args, │
│ │
│ /home/ToolBench-master/toolbench/model/model_adapter.py:21 in │
│ │
│ 18 │ AutoTokenizer, │
│ 19 │ LlamaForCausalLM, │
│ 20 ) │
│ ❱ 21 from peft import PeftModel │
│ 22 │
│ 23 from toolbench.tool_conversation import Conversation, get_conv_templat │
│ 24 from toolbench.model.compression import load_compress_model │
│ │
│ /home/anaconda3/envs/toolllm/lib/python3.11/site-packages/peft/init.py:2 │
│ 2 in │
│ │
│ 19 │
│ 20 version = "0.3.0" │
│ 21 │
│ ❱ 22 from .mapping import MODEL_TYPE_TO_PEFT_MODEL_MAPPING, PEFT_TYPE_TO_CON │
│ 23 from .peft_model import ( │
│ 24 │ PeftModel, │
│ 25 │ PeftModelForCausalLM, │
│ │
│ /home/anaconda3/envs/toolllm/lib/python3.11/site-packages/peft/mapping.py:16 │
│ in │
│ │
│ 13 # See the License for the specific language governing permissions and │
│ 14 # limitations under the License. │
│ 15 │
│ ❱ 16 from .peft_model import ( │
│ 17 │ PeftModel, │
│ 18 │ PeftModelForCausalLM, │
│ 19 │ PeftModelForSeq2SeqLM, │
│ │
│ /home/anaconda3/envs/toolllm/lib/python3.11/site-packages/peft/peft_model.py │
│ :31 in │
│ │
│ 28 from transformers.modeling_outputs import SequenceClassifierOutput, T │
│ 29 from transformers.utils import PushToHubMixin │
│ 30 │
│ ❱ 31 from .tuners import ( │
│ 32 │ AdaLoraModel, │
│ 33 │ AdaptionPromptModel, │
│ 34 │ LoraModel, │
│ │
│ /home/anaconda3/envs/toolllm/lib/python3.11/site-packages/peft/tuners/__init │
│ __.py:21 in │
│ │
│ 18 # limitations under the License. │
│ 19 │
│ 20 from .adaption_prompt import AdaptionPromptConfig, AdaptionPromptModel │
│ ❱ 21 from .lora import LoraConfig, LoraModel │
│ 22 from .adalora import AdaLoraConfig, AdaLoraModel │
│ 23 from .p_tuning import PromptEncoder, PromptEncoderConfig, PromptEncoder │
│ 24 from .prefix_tuning import PrefixEncoder, PrefixTuningConfig │
│ │
│ /home/anaconda3/envs/toolllm/lib/python3.11/site-packages/peft/tuners/lora.p │
│ y:40 in │
│ │
│ 37 │
│ 38 │
│ 39 if is_bnb_available(): │
│ ❱ 40 │ import bitsandbytes as bnb │
│ 41 │
│ 42 │
│ 43 @DataClass │
│ │
│ /home/anaconda3/envs/toolllm/lib/python3.11/site-packages/bitsandbytes-0.38. │
│ 0.post2-py3.11.egg/bitsandbytes/init.py:7 in │
│ │
│ 4 # LICENSE file in the root directory of this source tree. │
│ 5 │
│ 6 from . import cuda_setup, utils │
│ ❱ 7 from .autograd._functions import ( │
│ 8 │ MatmulLtState, │
│ 9 │ bmm_cublas, │
│ 10 │ matmul, │
│ │
│ /home/anaconda3/envs/toolllm/lib/python3.11/site-packages/bitsandbytes-0.38. │
│ 0.post2-py3.11.egg/bitsandbytes/autograd/init.py:1 in │
│ │
│ ❱ 1 from ._functions import undo_layout, get_inverse_transform_indices │
│ 2 │
│ │
│ /home/anaconda3/envs/toolllm/lib/python3.11/site-packages/bitsandbytes-0.38. │
│ 0.post2-py3.11.egg/bitsandbytes/autograd/_functions.py:9 in │
│ │
│ 6 │
│ 7 import torch │
│ 8 │
│ ❱ 9 import bitsandbytes.functional as F │
│ 10 │
│ 11 │
│ 12 # math.prod not compatible with python < 3.8 │
│ │
│ /home/anaconda3/envs/toolllm/lib/python3.11/site-packages/bitsandbytes-0.38. │
│ 0.post2-py3.11.egg/bitsandbytes/functional.py:17 in │
│ │
│ 14 from typing import Tuple │
│ 15 from torch import Tensor │
│ 16 │
│ ❱ 17 from .cextension import COMPILED_WITH_CUDA, lib │
│ 18 │
│ 19 │
│ 20 # math.prod not compatible with python < 3.8 │
│ │
│ /home/anaconda3/envs/toolllm/lib/python3.11/site-packages/bitsandbytes-0.38. │
│ 0.post2-py3.11.egg/bitsandbytes/cextension.py:20 in │
│ │
│ 17 │ if lib is None and torch.cuda.is_available(): │
│ 18 │ │ CUDASetup.get_instance().generate_instructions() │
│ 19 │ │ CUDASetup.get_instance().print_log_stack() │
│ ❱ 20 │ │ raise RuntimeError(''' │
│ 21 │ │ CUDA Setup failed despite GPU being available. Please run the f │
│ 22 │ │ │
│ 23 │ │ python -m bitsandbytes │
╰──────────────────────────────────────────────────────────────────────────────╯
RuntimeError:
CUDA Setup failed despite GPU being available. Please run the following
command to get more information:
libraries. You might need to add them
to your LD_LIBRARY_PATH. If you suspect a bug, please take the
information from python -m bitsandbytes
and open an issue at: https://github.com/TimDettmers/bitsandbytes/issues
./scripts/inference_toolllama_helloworld_demo.sh:行3: --tool_root_dir: 未找到命令
./scripts/inference_toolllama_helloworld_demo.sh:行4: --backbone_model: 未找到命令
./scripts/inference_toolllama_helloworld_demo.sh:行5: --model_path: 未找到命令
./scripts/inference_toolllama_helloworld_demo.sh:行6: --max_observation_length: 未找到命令
./scripts/inference_toolllama_helloworld_demo.sh:行7: --observ_compress_method: 未找到命令
./scripts/inference_toolllama_helloworld_demo.sh:行8: --method: 未找到命令
./scripts/inference_toolllama_helloworld_demo.sh:行9: --input_query_file: 未找到命令
./scripts/inference_toolllama_helloworld_demo.sh:行10: --output_answer_file: 未找到命令
./scripts/inference_toolllama_helloworld_demo.sh:行11: --api_customization: 未找到命令
终端输入nvcc -V
![微信图片_20240704153837](https://private-user-images.githubusercontent.com/56387272/345740036-76c35ca3-8f7b-43da-872e-5e35d56ac860.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjU3NzcwOTcsIm5iZiI6MTcyNTc3Njc5NywicGF0aCI6Ii81NjM4NzI3Mi8zNDU3NDAwMzYtNzZjMzVjYTMtOGY3Yi00M2RhLTg3MmUtNWUzNWQ1NmFjODYwLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA5MDglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwOTA4VDA2MjYzN1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTk1ZTM0NzlkYTY4Y2ExNjVhNjMwMDQ0YjE0ZTNhMDg1NDMwNzMyYzUwYjY0ZjNmNTY4MDAyYzdlYWYzZTczMDEmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.J1GiMw0IAM1Od9YO37Ug41jQU_5BaOmoryJ4hCCHPsU)
能看出来cuda是正确安装的,
找不到解决办法了55555555,特来问问大家,有知道怎么解决的嘛
The text was updated successfully, but these errors were encountered: