Run LLaMA and Alpaca on your computer.
This is a fork of Dalai to add a ChatGPT-style UI to it.
Also supports Alpaca 30B, you will need to download it manually from here
Please consider giving this project a star if you like it. It would motivate me to work on the fork and improve it further.
- 7B: ~5 GB
- 13B: ~9 GB
- 30B: ~21 GB
- 65B: ~42 GB
7B: 4.21GB (pre-quantized) 13B: 8.14GB (pre-quantized)
Note: after quantizing the model, you can delete the rest of the files and keep only the quantized model. 7B
- Full: The model takes up 31.17GB
- Quantized: 4.21GB
13B
- Full: The model takes up 60.21GB
- Quantized: 4.07GB * 2 = 8.14GB
30B
- Full: The model takes up 150.48GB
- Quantized: 5.09GB * 4 = 20.36GB
65B
- Full: The model takes up 432.64GB
- Quantized: 5.11GB * 8 = 40.88GB
- Install Homebrew
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
- Install dependencies with Homebrew
brew install cmake
brew install pkg-config
brew install git
brew install yarn
- Download Dalai-Pi
git clone https://github.com/ItsPi3141/dalai ~/dalai_pi
- Install required Nodejs modules
cd ~/dalai_pi
yarn install
- Download LLaMA or Alpaca models
Note: replace "llama" with "alpaca" if you want Alpaca. Change "7B" to the model size that you want. Alpaca only has 7B and 13B while LLaMA has 7B, 13B, 30B, and 65B.
npx dalai llama install 7B
- Start Dalai-Pi web UI
npx dalai serve
After the web UI has opened, go to https://localhost:42069 and enjoy!
- Install Visual Studio by downloading and running this script dalai-install-vs2019-only-win.bat
IMPORTANT
On Windows, make sure to run all commands in cmd.
DO NOT run in powershell. Powershell has unnecessarily strict permissions and makes the script fail silently.
-
Install Git from here
Note: Make sure you check the add to PATH option
-
Download Dalai-Pi
git clone https://github.com/ItsPi3141/dalai %USERPROFILE%\dalai_pi
- Install required Nodejs modules
cd %USERPROFILE%\dalai_pi
npm install yarn -g
yarn install
- Download LLaMA or Alpaca models
Note: replace "llama" with "alpaca" if you want Alpaca. Change "7B" to the model size that you want. Alpaca only has 7B and 13B while LLaMA has 7B, 13B, 30B, and 65B.
npx dalai llama install 7B
- Start Dalai-Pi web UI
.\webui-start.bat
- Install dependencies Ubuntu/Debian:
sudo apt update
sudo apt install build-essential python3-venv -y
Fedora/CentOS:
sudo dnf upgrade --refresh
sudo dnf install -y make automake gcc gcc-c++ kernel-devel python3-virtualenv
- Install Python and Node.js Ubuntu/Debian:
cd ~
curl -sL https://deb.nodesource.com/setup_18.x -o /tmp/nodesource_setup.sh
sudo bash /tmp/nodesource_setup.sh
sudo apt install nodejs python3 python-is-python3 -y
Fedora/CentOS:
sudo dnf install -y curl
curl -fsSL https://rpm.nodesource.com/setup_18.x | sudo bash -
sudo dnf install -y nodejs python python3-pip python-numpy
- Download Dalai-Pi
git clone https://github.com/ItsPi3141/dalai ~/dalai_pi
- Install required Nodejs modules
cd ~/dalai_pi
yarn install
- Download LLaMA or Alpaca models
Note: replace "llama" with "alpaca" if you want Alpaca. Change "7B" to the model size that you want. Alpaca only has 7B and 13B while LLaMA has 7B, 13B, 30B, and 65B.
npx dalai llama install 7B
- Start Dalai-Pi web UI
npx dalai serve
After the web UI has opened, go to https://localhost:42069 and enjoy!
Have questions or feedback? Follow the project through the following outlets: