Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

When selecting the GPT-4 model in the telegram-bot example settings, and sending a prompt, the bot throws an error #2641

Open
greygoo opened this issue Jun 23, 2024 · 1 comment
Open

When selecting the GPT-4 model in the telegram-bot example settings, and sending a prompt, the bot throws an error #2641

greygoo opened this issue Jun 23, 2024 · 1 comment
Labels
bug Something isn't working unconfirmed

Comments

@greygoo
Copy link
Contributor

greygoo commented Jun 23, 2024

LocalAI version:

quay.io/go-skynet/local-ai:v2.17.1-ffmpeg

Environment, CPU architecture, OS, and Version:

rtx4060/ryzen5700/32G

Describe the bug

When selecting the GPT-4 model in the telegram-bot example settings, and sending a prompt, the bot throws an error:

api-1                 | 4:02PM DBG Request received: {"model":"gpt-4","language":"","n":0,"top_p":1,"top_k":null,"temperature":0.7,"max_tokens":1000,"echo":false,"batch":0,"ignore_eos":false,"repeat_penalty":0,"n_keep":0,"frequency_penalty":0,"presence_penalty":0,"tfz":null,"typical_p":null,"seed":null,"negative_prompt":"","rope_freq_base":0,"rope_freq_scale":0,"negative_prompt_scale":0,"use_fast_tokenizer":false,"clip_skip":0,"tokenizer":"","file":"","size":"","prompt":null,"instruction":"","input":null,"stop":null,"messages":[{"role":"system","content":""},{"role":"user","content":"plan a trip from leipzig to berlin"}],"functions":null,"function_call":null,"stream":false,"mode":0,"step":0,"grammar":"","grammar_json_functions":null,"grammar_json_name":null,"backend":"","model_base_name":""}
api-1                 | 4:02PM DBG guessDefaultsFromFile: not a GGUF file
api-1                 | 4:02PM DBG Configuration read: &{PredictionOptions:{Model:gpt-4 Language: N:0 TopP:0xc0005d7a70 TopK:0xc0005d7b30 Temperature:0xc0005d7a60 Maxtokens:0xc0005d7a68 Echo:false Batch:0 IgnoreEOS:false RepeatPenalty:0 Keep:0 FrequencyPenalty:0 PresencePenalty:0 TFZ:0xc0005d7b60 TypicalP:0xc0005d7b58 Seed:0xc0005d7b80 NegativePrompt: RopeFreqBase:0 RopeFreqScale:0 NegativePromptScale:0 UseFastTokenizer:false ClipSkip:0 Tokenizer:} Name: F16:0xc0005d7b20 Threads:0xc0005d7b18 Debug:0xc0005d7b78 Roles:map[] Embeddings:false Backend: TemplateConfig:{Chat: ChatMessage: Completion: Edit: Functions: UseTokenizerTemplate:false JoinChatMessagesByCharacter:<nil>} PromptStrings:[] InputStrings:[] InputToken:[] functionCallString: functionCallNameString: ResponseFormat: ResponseFormatMap:map[] FunctionsConfig:{DisableNoAction:false GrammarConfig:{ParallelCalls:false DisableParallelNewLines:false MixedMode:false NoMixedFreeString:false NoGrammar:false Prefix: ExpectStringsAfterJSON:false} NoActionFunctionName: NoActionDescriptionName: ResponseRegex:[] JSONRegexMatch:[] ReplaceFunctionResults:[] ReplaceLLMResult:[] CaptureLLMResult:[] FunctionName:false} FeatureFlag:map[] LLMConfig:{SystemPrompt: TensorSplit: MainGPU: RMSNormEps:0 NGQA:0 PromptCachePath: PromptCacheAll:false PromptCacheRO:false MirostatETA:0xc0005d7b50 MirostatTAU:0xc0005d7b48 Mirostat:0xc0005d7b40 NGPULayers:0xc0005d7b70 MMap:0xc0005d7b78 MMlock:0xc0005d7b79 LowVRAM:0xc0005d7b79 Grammar: StopWords:[] Cutstrings:[] TrimSpace:[] TrimSuffix:[] ContextSize:0xc0005d7b10 NUMA:false LoraAdapter: LoraBase: LoraScale:0 NoMulMatQ:false DraftModel: NDraft:0 Quantization: GPUMemoryUtilization:0 TrustRemoteCode:false EnforceEager:false SwapSpace:0 MaxModelLen:0 TensorParallelSize:0 MMProj: FlashAttention:false NoKVOffloading:false RopeScaling: ModelType: YarnExtFactor:0 YarnAttnFactor:0 YarnBetaFast:0 YarnBetaSlow:0} AutoGPTQ:{ModelBaseName: Device: Triton:false UseFastTokenizer:false} Diffusers:{CUDA:false PipelineType: SchedulerType: EnableParameters: CFGScale:0 IMG2IMG:false ClipSkip:0 ClipModel: ClipSubFolder: ControlNet:} Step:0 GRPC:{Attempts:0 AttemptsSleepTime:0} TTSConfig:{Voice: VallE:{AudioPath:}} CUDA:false DownloadFiles:[] Description: Usage:}
api-1                 | 4:02PM DBG Parameters: &{PredictionOptions:{Model:gpt-4 Language: N:0 TopP:0xc0005d7a70 TopK:0xc0005d7b30 Temperature:0xc0005d7a60 Maxtokens:0xc0005d7a68 Echo:false Batch:0 IgnoreEOS:false RepeatPenalty:0 Keep:0 FrequencyPenalty:0 PresencePenalty:0 TFZ:0xc0005d7b60 TypicalP:0xc0005d7b58 Seed:0xc0005d7b80 NegativePrompt: RopeFreqBase:0 RopeFreqScale:0 NegativePromptScale:0 UseFastTokenizer:false ClipSkip:0 Tokenizer:} Name: F16:0xc0005d7b20 Threads:0xc0005d7b18 Debug:0xc0005d7b78 Roles:map[] Embeddings:false Backend: TemplateConfig:{Chat: ChatMessage: Completion: Edit: Functions: UseTokenizerTemplate:false JoinChatMessagesByCharacter:<nil>} PromptStrings:[] InputStrings:[] InputToken:[] functionCallString: functionCallNameString: ResponseFormat: ResponseFormatMap:map[] FunctionsConfig:{DisableNoAction:false GrammarConfig:{ParallelCalls:false DisableParallelNewLines:false MixedMode:false NoMixedFreeString:false NoGrammar:false Prefix: ExpectStringsAfterJSON:false} NoActionFunctionName: NoActionDescriptionName: ResponseRegex:[] JSONRegexMatch:[] ReplaceFunctionResults:[] ReplaceLLMResult:[] CaptureLLMResult:[] FunctionName:false} FeatureFlag:map[] LLMConfig:{SystemPrompt: TensorSplit: MainGPU: RMSNormEps:0 NGQA:0 PromptCachePath: PromptCacheAll:false PromptCacheRO:false MirostatETA:0xc0005d7b50 MirostatTAU:0xc0005d7b48 Mirostat:0xc0005d7b40 NGPULayers:0xc0005d7b70 MMap:0xc0005d7b78 MMlock:0xc0005d7b79 LowVRAM:0xc0005d7b79 Grammar: StopWords:[] Cutstrings:[] TrimSpace:[] TrimSuffix:[] ContextSize:0xc0005d7b10 NUMA:false LoraAdapter: LoraBase: LoraScale:0 NoMulMatQ:false DraftModel: NDraft:0 Quantization: GPUMemoryUtilization:0 TrustRemoteCode:false EnforceEager:false SwapSpace:0 MaxModelLen:0 TensorParallelSize:0 MMProj: FlashAttention:false NoKVOffloading:false RopeScaling: ModelType: YarnExtFactor:0 YarnAttnFactor:0 YarnBetaFast:0 YarnBetaSlow:0} AutoGPTQ:{ModelBaseName: Device: Triton:false UseFastTokenizer:false} Diffusers:{CUDA:false PipelineType: SchedulerType: EnableParameters: CFGScale:0 IMG2IMG:false ClipSkip:0 ClipModel: ClipSubFolder: ControlNet:} Step:0 GRPC:{Attempts:0 AttemptsSleepTime:0} TTSConfig:{Voice: VallE:{AudioPath:}} CUDA:false DownloadFiles:[] Description: Usage:}
api-1                 | 4:02PM DBG Prompt (before templating): 
api-1                 | plan a trip from leipzig to berlin
api-1                 | 4:02PM DBG Prompt (after templating): 
api-1                 | plan a trip from leipzig to berlin
api-1                 | 4:02PM DBG Loading from the following backends (in order): [llama-cpp llama-ggml gpt4all llama-cpp-fallback piper whisper rwkv stablediffusion huggingface bert-embeddings /build/backend/python/exllama/run.sh /build/backend/python/vall-e-x/run.sh /build/backend/python/exllama2/run.sh /build/backend/python/rerankers/run.sh /build/backend/python/mamba/run.sh /build/backend/python/transformers-musicgen/run.sh /build/backend/python/autogptq/run.sh /build/backend/python/openvoice/run.sh /build/backend/python/petals/run.sh /build/backend/python/diffusers/run.sh /build/backend/python/sentencetransformers/run.sh /build/backend/python/transformers/run.sh /build/backend/python/parler-tts/run.sh /build/backend/python/vllm/run.sh /build/backend/python/bark/run.sh /build/backend/python/coqui/run.sh /build/backend/python/sentencetransformers/run.sh]
api-1                 | 4:02PM INF Trying to load the model 'gpt-4' with the backend '[llama-cpp llama-ggml gpt4all llama-cpp-fallback piper whisper rwkv stablediffusion huggingface bert-embeddings /build/backend/python/exllama/run.sh /build/backend/python/vall-e-x/run.sh /build/backend/python/exllama2/run.sh /build/backend/python/rerankers/run.sh /build/backend/python/mamba/run.sh /build/backend/python/transformers-musicgen/run.sh /build/backend/python/autogptq/run.sh /build/backend/python/openvoice/run.sh /build/backend/python/petals/run.sh /build/backend/python/diffusers/run.sh /build/backend/python/sentencetransformers/run.sh /build/backend/python/transformers/run.sh /build/backend/python/parler-tts/run.sh /build/backend/python/vllm/run.sh /build/backend/python/bark/run.sh /build/backend/python/coqui/run.sh /build/backend/python/sentencetransformers/run.sh]'
api-1                 | 4:02PM INF [llama-cpp] Attempting to load
api-1                 | 4:02PM INF Loading model 'gpt-4' with backend llama-cpp
api-1                 | 4:02PM DBG Loading model in memory from file: /models/gpt-4
api-1                 | 4:02PM DBG Loading Model gpt-4 with gRPC (file: /models/gpt-4) (backend: llama-cpp): {backendString:llama-cpp model:gpt-4 threads:8 assetDir:/tmp/localai/backend_data context:{emptyCtx:{}} gRPCOptions:0xc000226248 externalBackends:map[autogptq:/build/backend/python/autogptq/run.sh bark:/build/backend/python/bark/run.sh coqui:/build/backend/python/coqui/run.sh diffusers:/build/backend/python/diffusers/run.sh exllama:/build/backend/python/exllama/run.sh exllama2:/build/backend/python/exllama2/run.sh huggingface-embeddings:/build/backend/python/sentencetransformers/run.sh mamba:/build/backend/python/mamba/run.sh openvoice:/build/backend/python/openvoice/run.sh parler-tts:/build/backend/python/parler-tts/run.sh petals:/build/backend/python/petals/run.sh rerankers:/build/backend/python/rerankers/run.sh sentencetransformers:/build/backend/python/sentencetransformers/run.sh transformers:/build/backend/python/transformers/run.sh transformers-musicgen:/build/backend/python/transformers-musicgen/run.sh vall-e-x:/build/backend/python/vall-e-x/run.sh vllm:/build/backend/python/vllm/run.sh] grpcAttempts:20 grpcAttemptsDelay:2 singleActiveBackend:false parallelRequests:false}
api-1                 | 4:02PM INF [llama-cpp] attempting to load with AVX2 variant
api-1                 | 4:02PM DBG Loading GRPC Process: /tmp/localai/backend_data/backend-assets/grpc/llama-cpp-avx2
api-1                 | 4:02PM DBG GRPC Service for gpt-4 will be running at: '127.0.0.1:42943'
api-1                 | 4:02PM DBG GRPC Service state dir: /tmp/go-processmanager3992485129
api-1                 | 4:02PM DBG GRPC Service Started
api-1                 | 4:02PM DBG GRPC(gpt-4-127.0.0.1:42943): stdout Server listening on 127.0.0.1:42943
api-1                 | 4:02PM DBG GRPC Service Ready
api-1                 | 4:02PM DBG GRPC: Loading model with options: {state:{NoUnkeyedLiterals:{} DoNotCompare:[] DoNotCopy:[] atomicMessageInfo:<nil>} sizeCache:0 unknownFields:[] Model:gpt-4 ContextSize:512 Seed:141500112 NBatch:512 F16Memory:false MLock:false MMap:true VocabOnly:false LowVRAM:false Embeddings:false NUMA:false NGPULayers:99999999 MainGPU: TensorSplit: Threads:8 LibrarySearchPath: RopeFreqBase:0 RopeFreqScale:0 RMSNormEps:0 NGQA:0 ModelFile:/models/gpt-4 Device: UseTriton:false ModelBaseName: UseFastTokenizer:false PipelineType: SchedulerType: CUDA:false CFGScale:0 IMG2IMG:false CLIPModel: CLIPSubfolder: CLIPSkip:0 ControlNet: Tokenizer: LoraBase: LoraAdapter: LoraScale:0 NoMulMatQ:false DraftModel: AudioPath: Quantization: GPUMemoryUtilization:0 TrustRemoteCode:false EnforceEager:false SwapSpace:0 MaxModelLen:0 TensorParallelSize:0 MMProj: RopeScaling: YarnExtFactor:0 YarnAttnFactor:0 YarnBetaFast:0 YarnBetaSlow:0 Type: FlashAttention:false NoKVOffload:false}
api-1                 | 4:02PM DBG GRPC(gpt-4-127.0.0.1:42943): stdout {"timestamp":1719158531,"level":"ERROR","function":"load_model","line":464,"message":"unable to load model","model":"/models/gpt-4"}
api-1                 | 4:02PM DBG GRPC(gpt-4-127.0.0.1:42943): stderr llama_model_load: error loading model: llama_model_loader: failed to load model from /models/gpt-4
api-1                 | 4:02PM DBG GRPC(gpt-4-127.0.0.1:42943): stderr 
api-1                 | 4:02PM DBG GRPC(gpt-4-127.0.0.1:42943): stderr llama_load_model_from_file: failed to load model
api-1                 | 4:02PM DBG GRPC(gpt-4-127.0.0.1:42943): stderr llama_init_from_gpt_params: error: failed to load model '/models/gpt-4'
api-1                 | 4:02PM INF [llama-cpp] Fails: could not load model: rpc error: code = Canceled desc = 
api-1                 | 4:02PM INF [llama-ggml] Attempting to load
api-1                 | 4:02PM INF Loading model 'gpt-4' with backend llama-ggml
api-1                 | 4:02PM DBG Loading model in memory from file: /models/gpt-4
api-1                 | 4:02PM DBG Loading Model gpt-4 with gRPC (file: /models/gpt-4) (backend: llama-ggml): {backendString:llama-ggml model:gpt-4 threads:8 assetDir:/tmp/localai/backend_data context:{emptyCtx:{}} gRPCOptions:0xc000226248 externalBackends:map[autogptq:/build/backend/python/autogptq/run.sh bark:/build/backend/python/bark/run.sh coqui:/build/backend/python/coqui/run.sh diffusers:/build/backend/python/diffusers/run.sh exllama:/build/backend/python/exllama/run.sh exllama2:/build/backend/python/exllama2/run.sh huggingface-embeddings:/build/backend/python/sentencetransformers/run.sh mamba:/build/backend/python/mamba/run.sh openvoice:/build/backend/python/openvoice/run.sh parler-tts:/build/backend/python/parler-tts/run.sh petals:/build/backend/python/petals/run.sh rerankers:/build/backend/python/rerankers/run.sh sentencetransformers:/build/backend/python/sentencetransformers/run.sh transformers:/build/backend/python/transformers/run.sh transformers-musicgen:/build/backend/python/transformers-musicgen/run.sh vall-e-x:/build/backend/python/vall-e-x/run.sh vllm:/build/backend/python/vllm/run.sh] grpcAttempts:20 grpcAttemptsDelay:2 singleActiveBackend:false parallelRequests:false}
api-1                 | 4:02PM DBG Loading GRPC Process: /tmp/localai/backend_data/backend-assets/grpc/llama-ggml
api-1                 | 4:02PM DBG GRPC Service for gpt-4 will be running at: '127.0.0.1:33899'
api-1                 | 4:02PM DBG GRPC Service state dir: /tmp/go-processmanager3960671244
api-1                 | 4:02PM DBG GRPC Service Started
api-1                 | 4:02PM DBG GRPC(gpt-4-127.0.0.1:33899): stderr 2024/06/23 16:02:11 gRPC Server listening at 127.0.0.1:33899
api-1                 | 4:02PM DBG GRPC Service Ready
api-1                 | 4:02PM DBG GRPC: Loading model with options: {state:{NoUnkeyedLiterals:{} DoNotCompare:[] DoNotCopy:[] atomicMessageInfo:<nil>} sizeCache:0 unknownFields:[] Model:gpt-4 ContextSize:512 Seed:141500112 NBatch:512 F16Memory:false MLock:false MMap:true VocabOnly:false LowVRAM:false Embeddings:false NUMA:false NGPULayers:99999999 MainGPU: TensorSplit: Threads:8 LibrarySearchPath: RopeFreqBase:0 RopeFreqScale:0 RMSNormEps:0 NGQA:0 ModelFile:/models/gpt-4 Device: UseTriton:false ModelBaseName: UseFastTokenizer:false PipelineType: SchedulerType: CUDA:false CFGScale:0 IMG2IMG:false CLIPModel: CLIPSubfolder: CLIPSkip:0 ControlNet: Tokenizer: LoraBase: LoraAdapter: LoraScale:0 NoMulMatQ:false DraftModel: AudioPath: Quantization: GPUMemoryUtilization:0 TrustRemoteCode:false EnforceEager:false SwapSpace:0 MaxModelLen:0 TensorParallelSize:0 MMProj: RopeScaling: YarnExtFactor:0 YarnAttnFactor:0 YarnBetaFast:0 YarnBetaSlow:0 Type: FlashAttention:false NoKVOffload:false}
api-1                 | 4:02PM DBG GRPC(gpt-4-127.0.0.1:33899): stderr create_gpt_params: loading model /models/gpt-4
api-1                 | 4:02PM DBG GRPC(gpt-4-127.0.0.1:33899): stderr error loading model: failed to open /models/gpt-4: No such file or directory
api-1                 | 4:02PM DBG GRPC(gpt-4-127.0.0.1:33899): stderr llama_load_model_from_file: failed to load model
api-1                 | 4:02PM DBG GRPC(gpt-4-127.0.0.1:33899): stderr llama_init_from_gpt_params: error: failed to load model '/models/gpt-4'
api-1                 | 4:02PM DBG GRPC(gpt-4-127.0.0.1:33899): stderr load_binding_model: error: unable to load model
api-1                 | 4:02PM INF [llama-ggml] Fails: could not load model: rpc error: code = Unknown desc = failed loading model
api-1                 | 4:02PM INF [gpt4all] Attempting to load
api-1                 | 4:02PM INF Loading model 'gpt-4' with backend gpt4all
api-1                 | 4:02PM DBG Loading model in memory from file: /models/gpt-4
api-1                 | 4:02PM DBG Loading Model gpt-4 with gRPC (file: /models/gpt-4) (backend: gpt4all): {backendString:gpt4all model:gpt-4 threads:8 assetDir:/tmp/localai/backend_data context:{emptyCtx:{}} gRPCOptions:0xc000226248 externalBackends:map[autogptq:/build/backend/python/autogptq/run.sh bark:/build/backend/python/bark/run.sh coqui:/build/backend/python/coqui/run.sh diffusers:/build/backend/python/diffusers/run.sh exllama:/build/backend/python/exllama/run.sh exllama2:/build/backend/python/exllama2/run.sh huggingface-embeddings:/build/backend/python/sentencetransformers/run.sh mamba:/build/backend/python/mamba/run.sh openvoice:/build/backend/python/openvoice/run.sh parler-tts:/build/backend/python/parler-tts/run.sh petals:/build/backend/python/petals/run.sh rerankers:/build/backend/python/rerankers/run.sh sentencetransformers:/build/backend/python/sentencetransformers/run.sh transformers:/build/backend/python/transformers/run.sh transformers-musicgen:/build/backend/python/transformers-musicgen/run.sh vall-e-x:/build/backend/python/vall-e-x/run.sh vllm:/build/backend/python/vllm/run.sh] grpcAttempts:20 grpcAttemptsDelay:2 singleActiveBackend:false parallelRequests:false}
api-1                 | 4:02PM DBG Loading GRPC Process: /tmp/localai/backend_data/backend-assets/grpc/gpt4all
api-1                 | 4:02PM DBG GRPC Service for gpt-4 will be running at: '127.0.0.1:37475'
api-1                 | 4:02PM DBG GRPC Service state dir: /tmp/go-processmanager1681212453
api-1                 | 4:02PM DBG GRPC Service Started
api-1                 | 4:02PM DBG GRPC(gpt-4-127.0.0.1:37475): stderr 2024/06/23 16:02:13 gRPC Server listening at 127.0.0.1:37475
api-1                 | 4:02PM DBG GRPC Service Ready
api-1                 | 4:02PM DBG GRPC: Loading model with options: {state:{NoUnkeyedLiterals:{} DoNotCompare:[] DoNotCopy:[] atomicMessageInfo:<nil>} sizeCache:0 unknownFields:[] Model:gpt-4 ContextSize:512 Seed:141500112 NBatch:512 F16Memory:false MLock:false MMap:true VocabOnly:false LowVRAM:false Embeddings:false NUMA:false NGPULayers:99999999 MainGPU: TensorSplit: Threads:8 LibrarySearchPath:/tmp/localai/backend_data/backend-assets/gpt4all RopeFreqBase:0 RopeFreqScale:0 RMSNormEps:0 NGQA:0 ModelFile:/models/gpt-4 Device: UseTriton:false ModelBaseName: UseFastTokenizer:false PipelineType: SchedulerType: CUDA:false CFGScale:0 IMG2IMG:false CLIPModel: CLIPSubfolder: CLIPSkip:0 ControlNet: Tokenizer: LoraBase: LoraAdapter: LoraScale:0 NoMulMatQ:false DraftModel: AudioPath: Quantization: GPUMemoryUtilization:0 TrustRemoteCode:false EnforceEager:false SwapSpace:0 MaxModelLen:0 TensorParallelSize:0 MMProj: RopeScaling: YarnExtFactor:0 YarnAttnFactor:0 YarnBetaFast:0 YarnBetaSlow:0 Type: FlashAttention:false NoKVOffload:false}
api-1                 | 4:02PM DBG GRPC(gpt-4-127.0.0.1:37475): stderr load_model: error 'No such file or directory'
api-1                 | 4:02PM INF [gpt4all] Fails: could not load model: rpc error: code = Unknown desc = failed loading model
api-1                 | 4:02PM INF [llama-cpp-fallback] Attempting to load
api-1                 | 4:02PM INF Loading model 'gpt-4' with backend llama-cpp-fallback
api-1                 | 4:02PM DBG Loading model in memory from file: /models/gpt-4
api-1                 | 4:02PM DBG Loading Model gpt-4 with gRPC (file: /models/gpt-4) (backend: llama-cpp-fallback): {backendString:llama-cpp-fallback model:gpt-4 threads:8 assetDir:/tmp/localai/backend_data context:{emptyCtx:{}} gRPCOptions:0xc000226248 externalBackends:map[autogptq:/build/backend/python/autogptq/run.sh bark:/build/backend/python/bark/run.sh coqui:/build/backend/python/coqui/run.sh diffusers:/build/backend/python/diffusers/run.sh exllama:/build/backend/python/exllama/run.sh exllama2:/build/backend/python/exllama2/run.sh huggingface-embeddings:/build/backend/python/sentencetransformers/run.sh mamba:/build/backend/python/mamba/run.sh openvoice:/build/backend/python/openvoice/run.sh parler-tts:/build/backend/python/parler-tts/run.sh petals:/build/backend/python/petals/run.sh rerankers:/build/backend/python/rerankers/run.sh sentencetransformers:/build/backend/python/sentencetransformers/run.sh transformers:/build/backend/python/transformers/run.sh transformers-musicgen:/build/backend/python/transformers-musicgen/run.sh vall-e-x:/build/backend/python/vall-e-x/run.sh vllm:/build/backend/python/vllm/run.sh] grpcAttempts:20 grpcAttemptsDelay:2 singleActiveBackend:false parallelRequests:false}
api-1                 | 4:02PM DBG Loading GRPC Process: /tmp/localai/backend_data/backend-assets/grpc/llama-cpp-fallback
api-1                 | 4:02PM DBG GRPC Service for gpt-4 will be running at: '127.0.0.1:32827'
api-1                 | 4:02PM DBG GRPC Service state dir: /tmp/go-processmanager3771195248
api-1                 | 4:02PM DBG GRPC Service Started
api-1                 | 4:02PM DBG GRPC(gpt-4-127.0.0.1:32827): stdout Server listening on 127.0.0.1:32827
api-1                 | 4:02PM DBG GRPC Service Ready
api-1                 | 4:02PM DBG GRPC: Loading model with options: {state:{NoUnkeyedLiterals:{} DoNotCompare:[] DoNotCopy:[] atomicMessageInfo:<nil>} sizeCache:0 unknownFields:[] Model:gpt-4 ContextSize:512 Seed:141500112 NBatch:512 F16Memory:false MLock:false MMap:true VocabOnly:false LowVRAM:false Embeddings:false NUMA:false NGPULayers:99999999 MainGPU: TensorSplit: Threads:8 LibrarySearchPath:/tmp/localai/backend_data/backend-assets/gpt4all RopeFreqBase:0 RopeFreqScale:0 RMSNormEps:0 NGQA:0 ModelFile:/models/gpt-4 Device: UseTriton:false ModelBaseName: UseFastTokenizer:false PipelineType: SchedulerType: CUDA:false CFGScale:0 IMG2IMG:false CLIPModel: CLIPSubfolder: CLIPSkip:0 ControlNet: Tokenizer: LoraBase: LoraAdapter: LoraScale:0 NoMulMatQ:false DraftModel: AudioPath: Quantization: GPUMemoryUtilization:0 TrustRemoteCode:false EnforceEager:false SwapSpace:0 MaxModelLen:0 TensorParallelSize:0 MMProj: RopeScaling: YarnExtFactor:0 YarnAttnFactor:0 YarnBetaFast:0 YarnBetaSlow:0 Type: FlashAttention:false NoKVOffload:false}
api-1                 | 4:02PM DBG GRPC(gpt-4-127.0.0.1:32827): stdout {"timestamp":1719158537,"level":"ERROR","function":"load_model","line":464,"message":"unable to load model","model":"/models/gpt-4"}
api-1                 | 4:02PM DBG GRPC(gpt-4-127.0.0.1:32827): stderr llama_model_load: error loading model: llama_model_loader: failed to load model from /models/gpt-4
api-1                 | 4:02PM DBG GRPC(gpt-4-127.0.0.1:32827): stderr 
api-1                 | 4:02PM DBG GRPC(gpt-4-127.0.0.1:32827): stderr llama_load_model_from_file: failed to load model
api-1                 | 4:02PM DBG GRPC(gpt-4-127.0.0.1:32827): stderr llama_init_from_gpt_params: error: failed to load model '/models/gpt-4'
api-1                 | 4:02PM INF [llama-cpp-fallback] Fails: could not load model: rpc error: code = Canceled desc = 
api-1                 | 4:02PM INF [piper] Attempting to load
api-1                 | 4:02PM INF Loading model 'gpt-4' with backend piper
api-1                 | 4:02PM DBG Loading model in memory from file: /models/gpt-4
api-1                 | 4:02PM DBG Loading Model gpt-4 with gRPC (file: /models/gpt-4) (backend: piper): {backendString:piper model:gpt-4 threads:8 assetDir:/tmp/localai/backend_data context:{emptyCtx:{}} gRPCOptions:0xc000226248 externalBackends:map[autogptq:/build/backend/python/autogptq/run.sh bark:/build/backend/python/bark/run.sh coqui:/build/backend/python/coqui/run.sh diffusers:/build/backend/python/diffusers/run.sh exllama:/build/backend/python/exllama/run.sh exllama2:/build/backend/python/exllama2/run.sh huggingface-embeddings:/build/backend/python/sentencetransformers/run.sh mamba:/build/backend/python/mamba/run.sh openvoice:/build/backend/python/openvoice/run.sh parler-tts:/build/backend/python/parler-tts/run.sh petals:/build/backend/python/petals/run.sh rerankers:/build/backend/python/rerankers/run.sh sentencetransformers:/build/backend/python/sentencetransformers/run.sh transformers:/build/backend/python/transformers/run.sh transformers-musicgen:/build/backend/python/transformers-musicgen/run.sh vall-e-x:/build/backend/python/vall-e-x/run.sh vllm:/build/backend/python/vllm/run.sh] grpcAttempts:20 grpcAttemptsDelay:2 singleActiveBackend:false parallelRequests:false}
...
chatgpt_telegram_bot  | [/build/backend/python/autogptq/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/autogptq/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS
chatgpt_telegram_bot  | [/build/backend/python/openvoice/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/openvoice/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS
chatgpt_telegram_bot  | [/build/backend/python/petals/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/petals/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS
chatgpt_telegram_bot  | [/build/backend/python/diffusers/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/diffusers/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS
chatgpt_telegram_bot  | [/build/backend/python/sentencetransformers/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/sentencetransformers/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS
chatgpt_telegram_bot  | [/build/backend/python/transformers/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/transformers/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS
chatgpt_telegram_bot  | [/build/backend/python/parler-tts/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/parler-tts/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS
chatgpt_telegram_bot  | [/build/backend/python/vllm/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/vllm/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS
chatgpt_telegram_bot  | [/build/backend/python/bark/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/bark/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS
chatgpt_telegram_bot  | [/build/backend/python/coqui/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/coqui/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS
chatgpt_telegram_bot  | [/build/backend/python/sentencetransformers/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/sentencetransformers/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS {"error":{"code":500,"message":"could not load model - all backends returned error: [llama-cpp]: could not load model: rpc error: code = Canceled desc = \n[llama-ggml]: could not load model: rpc error: code = Unknown desc = failed loading model\n[gpt4all]: could not load model: rpc error: code = Unknown desc = failed loading model\n[llama-cpp-fallback]: could not load model: rpc error: code = Canceled desc = \n[piper]: could not load model: rpc error: code = Unknown desc = unsupported model type /models/gpt-4 (should end with .onnx)\n[whisper]: could not load model: rpc error: code = Unknown desc = stat /models/gpt-4: no such file or directory\n[rwkv]: could not load model: rpc error: code = Unavailable desc = error reading from server: EOF\n[stablediffusion]: could not load model: rpc error: code = Unknown desc = stat /models/gpt-4: no such file or directory\n[huggingface]: could not load model: rpc error: code = Unknown desc = no huggingface token provided\n[bert-embeddings]: could not load model: rpc error: code = Unknown desc = failed loading model\n[/build/backend/python/exllama/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/exllama/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/vall-e-x/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/vall-e-x/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/exllama2/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/exllama2/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/rerankers/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/rerankers/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/mamba/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/mamba/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/transformers-musicgen/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/transformers-musicgen/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/autogptq/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/autogptq/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/openvoice/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/openvoice/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/petals/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/petals/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/diffusers/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/diffusers/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/sentencetransformers/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/sentencetransformers/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/transformers/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/transformers/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/parler-tts/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/parler-tts/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/vllm/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/vllm/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/bark/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/bark/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/coqui/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/coqui/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/sentencetransformers/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/sentencetransformers/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS","type":""}} 500 {'error': {'code': 500, 'message': 'could not load model - all backends returned error: [llama-cpp]: could not load model: rpc error: code = Canceled desc = \n[llama-ggml]: could not load model: rpc error: code = Unknown desc = failed loading model\n[gpt4all]: could not load model: rpc error: code = Unknown desc = failed loading model\n[llama-cpp-fallback]: could not load model: rpc error: code = Canceled desc = \n[piper]: could not load model: rpc error: code = Unknown desc = unsupported model type /models/gpt-4 (should end with .onnx)\n[whisper]: could not load model: rpc error: code = Unknown desc = stat /models/gpt-4: no such file or directory\n[rwkv]: could not load model: rpc error: code = Unavailable desc = error reading from server: EOF\n[stablediffusion]: could not load model: rpc error: code = Unknown desc = stat /models/gpt-4: no such file or directory\n[huggingface]: could not load model: rpc error: code = Unknown desc = no huggingface token provided\n[bert-embeddings]: could not load model: rpc error: code = Unknown desc = failed loading model\n[/build/backend/python/exllama/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/exllama/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/vall-e-x/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/vall-e-x/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/exllama2/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/exllama2/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/rerankers/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/rerankers/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/mamba/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/mamba/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/transformers-musicgen/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/transformers-musicgen/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/autogptq/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/autogptq/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/openvoice/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/openvoice/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/petals/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/petals/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/diffusers/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/diffusers/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/sentencetransformers/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/sentencetransformers/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/transformers/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/transformers/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/parler-tts/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/parler-tts/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/vllm/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/vllm/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/bark/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/bark/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/coqui/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/coqui/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/sentencetransformers/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/sentencetransformers/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS', 'type': ''}} <CIMultiDictProxy('Date': 'Sun, 23 Jun 2024 16:02:29 GMT', 'Content-Type': 'application/json', 'Content-Length': '4947')>
api-1                 | 4:02PM INF Success ip=127.0.0.1 latency="22.908µs" method=GET status=200 url=/readyz
chatgpt_telegram_bot  | Exception while handling an update:
chatgpt_telegram_bot  | Traceback (most recent call last):
chatgpt_telegram_bot  |   File "/code/bot/bot.py", line 351, in message_handle_fn
chatgpt_telegram_bot  |     answer, (n_input_tokens, n_output_tokens), n_first_dialog_messages_removed = await chatgpt_instance.send_message(
chatgpt_telegram_bot  |                                                                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
chatgpt_telegram_bot  |   File "/code/bot/openai_utils.py", line 40, in send_message
chatgpt_telegram_bot  |     r = await openai.ChatCompletion.acreate(
chatgpt_telegram_bot  |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
chatgpt_telegram_bot  |   File "/usr/local/lib/python3.11/site-packages/openai/api_resources/chat_completion.py", line 45, in acreate
chatgpt_telegram_bot  |     return await super().acreate(*args, **kwargs)
chatgpt_telegram_bot  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
chatgpt_telegram_bot  |   File "/usr/local/lib/python3.11/site-packages/openai/api_resources/abstract/engine_api_resource.py", line 217, in acreate
chatgpt_telegram_bot  |     response, _, api_key = await requestor.arequest(
chatgpt_telegram_bot  |                            ^^^^^^^^^^^^^^^^^^^^^^^^^
chatgpt_telegram_bot  |   File "/usr/local/lib/python3.11/site-packages/openai/api_requestor.py", line 382, in arequest
chatgpt_telegram_bot  |     resp, got_stream = await self._interpret_async_response(result, stream)
chatgpt_telegram_bot  |                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
chatgpt_telegram_bot  |   File "/usr/local/lib/python3.11/site-packages/openai/api_requestor.py", line 728, in _interpret_async_response
chatgpt_telegram_bot  |     self._interpret_response_line(
chatgpt_telegram_bot  |   File "/usr/local/lib/python3.11/site-packages/openai/api_requestor.py", line 765, in _interpret_response_line
chatgpt_telegram_bot  |     raise self.handle_error_response(
chatgpt_telegram_bot  | openai.error.APIError: could not load model - all backends returned error: [llama-cpp]: could not load model: rpc error: code = Canceled desc = 
chatgpt_telegram_bot  | [llama-ggml]: could not load model: rpc error: code = Unknown desc = failed loading model
chatgpt_telegram_bot  | [gpt4all]: could not load model: rpc error: code = Unknown desc = failed loading model
chatgpt_telegram_bot  | [llama-cpp-fallback]: could not load model: rpc error: code = Canceled desc = 
chatgpt_telegram_bot  | [piper]: could not load model: rpc error: code = Unknown desc = unsupported model type /models/gpt-4 (should end with .onnx)
chatgpt_telegram_bot  | [whisper]: could not load model: rpc error: code = Unknown desc = stat /models/gpt-4: no such file or directory
chatgpt_telegram_bot  | [rwkv]: could not load model: rpc error: code = Unavailable desc = error reading from server: EOF
chatgpt_telegram_bot  | [stablediffusion]: could not load model: rpc error: code = Unknown desc = stat /models/gpt-4: no such file or directory
chatgpt_telegram_bot  | [huggingface]: could not load model: rpc error: code = Unknown desc = no huggingface token provided
chatgpt_telegram_bot  | [bert-embeddings]: could not load model: rpc error: code = Unknown desc = failed loading model
chatgpt_telegram_bot  | [/build/backend/python/exllama/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/exllama/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS
chatgpt_telegram_bot  | [/build/backend/python/vall-e-x/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/vall-e-x/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS
chatgpt_telegram_bot  | [/build/backend/python/exllama2/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/exllama2/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS
chatgpt_telegram_bot  | [/build/backend/python/rerankers/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/rerankers/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS
chatgpt_telegram_bot  | [/build/backend/python/mamba/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/mamba/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS
chatgpt_telegram_bot  | [/build/backend/python/transformers-musicgen/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/transformers-musicgen/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS
chatgpt_telegram_bot  | [/build/backend/python/autogptq/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/autogptq/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS
chatgpt_telegram_bot  | [/build/backend/python/openvoice/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/openvoice/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS
chatgpt_telegram_bot  | [/build/backend/python/petals/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/petals/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS
chatgpt_telegram_bot  | [/build/backend/python/diffusers/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/diffusers/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS
chatgpt_telegram_bot  | [/build/backend/python/sentencetransformers/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/sentencetransformers/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS
chatgpt_telegram_bot  | [/build/backend/python/transformers/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/transformers/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS
chatgpt_telegram_bot  | [/build/backend/python/parler-tts/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/parler-tts/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS
chatgpt_telegram_bot  | [/build/backend/python/vllm/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/vllm/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS
chatgpt_telegram_bot  | [/build/backend/python/bark/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/bark/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS
chatgpt_telegram_bot  | [/build/backend/python/coqui/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/coqui/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS
chatgpt_telegram_bot  | [/build/backend/python/sentencetransformers/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/sentencetransformers/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS {"error":{"code":500,"message":"could not load model - all backends returned error: [llama-cpp]: could not load model: rpc error: code = Canceled desc = \n[llama-ggml]: could not load model: rpc error: code = Unknown desc = failed loading model\n[gpt4all]: could not load model: rpc error: code = Unknown desc = failed loading model\n[llama-cpp-fallback]: could not load model: rpc error: code = Canceled desc = \n[piper]: could not load model: rpc error: code = Unknown desc = unsupported model type /models/gpt-4 (should end with .onnx)\n[whisper]: could not load model: rpc error: code = Unknown desc = stat /models/gpt-4: no such file or directory\n[rwkv]: could not load model: rpc error: code = Unavailable desc = error reading from server: EOF\n[stablediffusion]: could not load model: rpc error: code = Unknown desc = stat /models/gpt-4: no such file or directory\n[huggingface]: could not load model: rpc error: code = Unknown desc = no huggingface token provided\n[bert-embeddings]: could not load model: rpc error: code = Unknown desc = failed loading model\n[/build/backend/python/exllama/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/exllama/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/vall-e-x/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/vall-e-x/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/exllama2/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/exllama2/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/rerankers/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/rerankers/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/mamba/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/mamba/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/transformers-musicgen/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/transformers-musicgen/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/autogptq/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/autogptq/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/openvoice/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/openvoice/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/petals/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/petals/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/diffusers/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/diffusers/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/sentencetransformers/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/sentencetransformers/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/transformers/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/transformers/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/parler-tts/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/parler-tts/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/vllm/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/vllm/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/bark/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/bark/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/coqui/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/coqui/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/sentencetransformers/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/sentencetransformers/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS","type":""}} 500 {'error': {'code': 500, 'message': 'could not load model - all backends returned error: [llama-cpp]: could not load model: rpc error: code = Canceled desc = \n[llama-ggml]: could not load model: rpc error: code = Unknown desc = failed loading model\n[gpt4all]: could not load model: rpc error: code = Unknown desc = failed loading model\n[llama-cpp-fallback]: could not load model: rpc error: code = Canceled desc = \n[piper]: could not load model: rpc error: code = Unknown desc = unsupported model type /models/gpt-4 (should end with .onnx)\n[whisper]: could not load model: rpc error: code = Unknown desc = stat /models/gpt-4: no such file or directory\n[rwkv]: could not load model: rpc error: code = Unavailable desc = error reading from server: EOF\n[stablediffusion]: could not load model: rpc error: code = Unknown desc = stat /models/gpt-4: no such file or directory\n[huggingface]: could not load model: rpc error: code = Unknown desc = no huggingface token provided\n[bert-embeddings]: could not load model: rpc error: code = Unknown desc = failed loading model\n[/build/backend/python/exllama/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/exllama/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/vall-e-x/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/vall-e-x/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/exllama2/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/exllama2/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/rerankers/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/rerankers/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/mamba/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/mamba/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/transformers-musicgen/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/transformers-musicgen/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/autogptq/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/autogptq/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/openvoice/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/openvoice/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/petals/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/petals/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/diffusers/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/diffusers/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/sentencetransformers/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/sentencetransformers/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/transformers/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/transformers/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/parler-tts/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/parler-tts/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/vllm/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/vllm/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/bark/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/bark/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/coqui/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/coqui/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS\n[/build/backend/python/sentencetransformers/run.sh]: grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/build/backend/python/sentencetransformers/run.sh. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS', 'type': ''}} <CIMultiDictProxy('Date': 'Sun, 23 Jun 2024 16:02:29 GMT', 'Content-Type': 'application/json', 'Content-Length': '4947')>
chatgpt_telegram_bot  | 
chatgpt_telegram_bot  | During handling of the above exception, another exception occurred:
chatgpt_telegram_bot  | 
chatgpt_telegram_bot  | Traceback (most recent call last):
chatgpt_telegram_bot  |   File "/usr/local/lib/python3.11/site-packages/telegram/ext/_application.py", line 1104, in process_update
chatgpt_telegram_bot  |     await coroutine
chatgpt_telegram_bot  |   File "/usr/local/lib/python3.11/site-packages/telegram/ext/_handler.py", line 141, in handle_update
chatgpt_telegram_bot  |     return await self.callback(update, context)
chatgpt_telegram_bot  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
chatgpt_telegram_bot  |   File "/code/bot/bot.py", line 418, in message_handle
chatgpt_telegram_bot  |     await task
chatgpt_telegram_bot  |   File "/code/bot/bot.py", line 402, in message_handle_fn
chatgpt_telegram_bot  |     await update.message.reply_text(error_text)
chatgpt_telegram_bot  |   File "/usr/local/lib/python3.11/site-packages/telegram/_message.py", line 1041, in reply_text
chatgpt_telegram_bot  |     return await self.get_bot().send_message(
chatgpt_telegram_bot  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
chatgpt_telegram_bot  |   File "/usr/local/lib/python3.11/site-packages/telegram/ext/_extbot.py", line 2598, in send_message
chatgpt_telegram_bot  |     return await super().send_message(
chatgpt_telegram_bot  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
chatgpt_telegram_bot  |   File "/usr/local/lib/python3.11/site-packages/telegram/_bot.py", line 331, in decorator
chatgpt_telegram_bot  |     result = await func(*args, **kwargs)  # skipcq: PYL-E1102
chatgpt_telegram_bot  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
chatgpt_telegram_bot  |   File "/usr/local/lib/python3.11/site-packages/telegram/_bot.py", line 760, in send_message
chatgpt_telegram_bot  |     return await self._send_message(
chatgpt_telegram_bot  |            ^^^^^^^^^^^^^^^^^^^^^^^^^
chatgpt_telegram_bot  |   File "/usr/local/lib/python3.11/site-packages/telegram/ext/_extbot.py", line 488, in _send_message
chatgpt_telegram_bot  |     result = await super()._send_message(
chatgpt_telegram_bot  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
chatgpt_telegram_bot  |   File "/usr/local/lib/python3.11/site-packages/telegram/_bot.py", line 512, in _send_message
chatgpt_telegram_bot  |     result = await self._post(
chatgpt_telegram_bot  |              ^^^^^^^^^^^^^^^^^
chatgpt_telegram_bot  |   File "/usr/local/lib/python3.11/site-packages/telegram/_bot.py", line 419, in _post
chatgpt_telegram_bot  |     return await self._do_post(
chatgpt_telegram_bot  |            ^^^^^^^^^^^^^^^^^^^^
chatgpt_telegram_bot  |   File "/usr/local/lib/python3.11/site-packages/telegram/ext/_extbot.py", line 326, in _do_post
chatgpt_telegram_bot  |     return await self.rate_limiter.process_request(
chatgpt_telegram_bot  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
chatgpt_telegram_bot  |   File "/usr/local/lib/python3.11/site-packages/telegram/ext/_aioratelimiter.py", line 247, in process_request
chatgpt_telegram_bot  |     return await self._run_request(
chatgpt_telegram_bot  |            ^^^^^^^^^^^^^^^^^^^^^^^^
chatgpt_telegram_bot  |   File "/usr/local/lib/python3.11/site-packages/telegram/ext/_aioratelimiter.py", line 203, in _run_request
chatgpt_telegram_bot  |     return await callback(*args, **kwargs)
chatgpt_telegram_bot  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
chatgpt_telegram_bot  |   File "/usr/local/lib/python3.11/site-packages/telegram/_bot.py", line 450, in _do_post
chatgpt_telegram_bot  |     return await request.post(
chatgpt_telegram_bot  |            ^^^^^^^^^^^^^^^^^^^
chatgpt_telegram_bot  |   File "/usr/local/lib/python3.11/site-packages/telegram/request/_baserequest.py", line 165, in post
chatgpt_telegram_bot  |     result = await self._request_wrapper(
chatgpt_telegram_bot  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
chatgpt_telegram_bot  |   File "/usr/local/lib/python3.11/site-packages/telegram/request/_baserequest.py", line 328, in _request_wrapper
chatgpt_telegram_bot  |     raise BadRequest(message)
chatgpt_telegram_bot  | telegram.error.BadRequest: Message is too long

To Reproduce

  • run the commands in the telegram-bot example to start the bot
  • in telegram, use the /settings command to get to the language model settings
  • click on GPT-4
  • give a prompt
    => see the bot report a error in telegram

Expected behavior

A reply is given to the prompt

Logs

In description

Additional context
@greygoo greygoo added bug Something isn't working unconfirmed labels Jun 23, 2024
@greygoo
Copy link
Contributor Author

greygoo commented Jun 23, 2024

This can be fixed by copying aio/cpu/text-to-text.yaml to the models dir. It then downloads the required model and GPT-4 works in the bot as expected. Is there an easy way to add that to the preload section? If not, I'd suggest adding this step to the documentation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working unconfirmed
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@greygoo