You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Jun 24, 2024. It is now read-only.
Inference with both ggml and ggmf is broken on macOS with #107. It generates garbage:
llama-rs # head -c 4 ../llama-models/ggml-alpaca-7b-q4.bin
lmgg
# previous commit
llama-rs # git log --oneline -n 1
6dde2b6 (HEAD) Merge pull request #100 from rustformers/document_working_models
llama-rs # RUST_LOG=error cargo run --release --bin llama-cli -- -m ../llama-models/ggml-alpaca-7b-q4.bin -p "You are a son of a " --top-k 1
Finished release [optimized] target(s) in 0.14s
Running `target/release/llama-cli -m ../llama-models/ggml-alpaca-7b-q4.bin -p 'You are a son of a ' --top-k 1`
You are a son of a 70 year old man who has been diagnosed with terminal cancer. He is in the process of making his will and^C
# main
llama-rs # git checkout main
Previous HEAD position was 6dde2b6 Merge pull request #100 from rustformers/document_working_models
Switched to branch 'main'
Your branch is up to date with 'upstream/main'.
llama-rs # git log --oneline -n 1
f103ae9 (HEAD -> main, upstream/main) Merge pull request #104 from rustformers/fix/num_cpu_cli_arg
llama-rs # RUST_LOG=error cargo run --release --bin llama-cli -- -m ../llama-models/ggml-alpaca-7b-q4.bin -p "You are a son of a " --top-k 1
Finished release [optimized] target(s) in 4.27s
Running `target/release/llama-cli -m ../llama-models/ggml-alpaca-7b-q4.bin -p 'You are a son of a ' --top-k 1`
You are a son of a , and. (! R Al to in/ ///-0 E)[ c. that…7 C. your you from6Al. , t. …
- – & back2M !!!!!!!^C
This seems to work fine on Windows x86-64 (me) and Linux x86-64 (setzer), but macOS M1 (me and tonloc on the Discord) have garbage generated.
The text was updated successfully, but these errors were encountered:
Inference with both
ggml
andggmf
is broken on macOS with #107. It generates garbage:This seems to work fine on Windows x86-64 (me) and Linux x86-64 (setzer), but macOS M1 (me and tonloc on the Discord) have garbage generated.
The text was updated successfully, but these errors were encountered: