Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
CoderLSF committed Nov 16, 2023
1 parent c81b254 commit b3328c0
Showing 1 changed file with 4 additions and 3 deletions.
7 changes: 4 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,8 @@ See [llama2.c](https://github.com/karpathy/llama2.c)
./main -c ./models/stories110M.bin -z ./models/tokenizer.bin -j 14 -q int8 -n 200 -i 'That was a long long story happened in the ancient China.'
```

<img width="1501" alt="image" src="https://github.com/CoderLSF/fast-llama/assets/65639063/28156af1-142e-417c-9b94-8d931fac8884">

#### 2. Run with hugging face format models
**`Step 1`**: Download a model

Expand All @@ -74,6 +76,8 @@ python3 ./tools/convert_flm.py -m /path/to/model-directory -o ./models/model-nam
./main -c ./models/model-name-int8.flm -j 40 -n 200 -i 'That was a long long story happened in the ancient China.'
```

![image](https://github.com/CoderLSF/RapidLLaMA/assets/65639063/d4477fcb-96fb-4b0a-a1fd-30ca583d0aa2)

>
All supported command-line options are as follows:

Expand Down Expand Up @@ -114,9 +118,6 @@ Core(s) per socket: 28
Socket(s): 2
```

<img width="1501" alt="image" src="https://github.com/CoderLSF/fast-llama/assets/65639063/28156af1-142e-417c-9b94-8d931fac8884">

![image](https://github.com/CoderLSF/RapidLLaMA/assets/65639063/d4477fcb-96fb-4b0a-a1fd-30ca583d0aa2)

![2a58bda471f0aa2770f349dba73a530d](https://github.com/CoderLSF/fast-llama/assets/65639063/c3634948-280d-47c8-b9e7-ff07d7104b86)

Expand Down

0 comments on commit b3328c0

Please sign in to comment.