Run LLaMA and Alpaca on your computer.

GitHub Twitter Discord

This is a fork of Dalai to add a ChatGPT-style UI to it.
Also supports Alpaca 30B, you will need to download it manually from here

Please consider giving this project a star if you like it. It would motivate me to work on the fork and improve it further.

• 7B: ~5 GB
• 13B: ~9 GB
• 30B: ~21 GB
• 65B: ~42 GB

7B: 4.21GB (pre-quantized) 13B: 8.14GB (pre-quantized)

Note: after quantizing the model, you can delete the rest of the files and keep only the quantized model. 7B

• Full: The model takes up 31.17GB
• Quantized: 4.21GB

13B

• Full: The model takes up 60.21GB
• Quantized: 4.07GB * 2 = 8.14GB

30B

• Full: The model takes up 150.48GB
• Quantized: 5.09GB * 4 = 20.36GB

65B

• Full: The model takes up 432.64GB
• Quantized: 5.11GB * 8 = 40.88GB

1. Install Homebrew

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

2. Install dependencies with Homebrew

brew install cmake
brew install pkg-config
brew install git
brew install yarn

3. Download Dalai-Pi

git clone https://github.com/ItsPi3141/dalai ~/dalai_pi

4. Install required Nodejs modules

cd ~/dalai_pi
yarn install

5. Download LLaMA or Alpaca models

   Note: replace "llama" with "alpaca" if you want Alpaca. Change "7B" to the model size that you want. Alpaca only has 7B and 13B while LLaMA has 7B, 13B, 30B, and 65B.

npx dalai llama install 7B

6. Start Dalai-Pi web UI

npx dalai serve

After the web UI has opened, go to https://localhost:42069 and enjoy!

1. Install Visual Studio by downloading and running this script dalai-install-vs2019-only-win.bat

IMPORTANT
On Windows, make sure to run all commands in cmd.
DO NOT run in powershell. Powershell has unnecessarily strict permissions and makes the script fail silently.

2. Install Git from here

   Note: Make sure you check the add to PATH option

3. Download Dalai-Pi

git clone https://github.com/ItsPi3141/dalai %USERPROFILE%\dalai_pi

4. Install required Nodejs modules

cd %USERPROFILE%\dalai_pi
npm install yarn -g
yarn install

5. Download LLaMA or Alpaca models

   Note: replace "llama" with "alpaca" if you want Alpaca. Change "7B" to the model size that you want. Alpaca only has 7B and 13B while LLaMA has 7B, 13B, 30B, and 65B.

npx dalai llama install 7B

6. Start Dalai-Pi web UI

.\webui-start.bat

1. Install dependencies Ubuntu/Debian:

sudo apt update
sudo apt install build-essential python3-venv -y

Fedora/CentOS:

sudo dnf upgrade --refresh
sudo dnf install -y make automake gcc gcc-c++ kernel-devel python3-virtualenv

2. Install Python and Node.js Ubuntu/Debian:

cd ~
curl -sL https://deb.nodesource.com/setup_18.x -o /tmp/nodesource_setup.sh
sudo bash /tmp/nodesource_setup.sh
sudo apt install nodejs python3 python-is-python3 -y

Fedora/CentOS:

sudo dnf install -y curl
curl -fsSL https://rpm.nodesource.com/setup_18.x | sudo bash -
sudo dnf install -y nodejs python python3-pip python-numpy

3. Download Dalai-Pi

git clone https://github.com/ItsPi3141/dalai ~/dalai_pi

4. Install required Nodejs modules

cd ~/dalai_pi
yarn install

5. Download LLaMA or Alpaca models

   Note: replace "llama" with "alpaca" if you want Alpaca. Change "7B" to the model size that you want. Alpaca only has 7B and 13B while LLaMA has 7B, 13B, 30B, and 65B.

npx dalai llama install 7B

6. Start Dalai-Pi web UI

npx dalai serve

After the web UI has opened, go to https://localhost:42069 and enjoy!

Have questions or feedback? Follow the project through the following outlets:

GitHub Twitter Discord