For low GPU VRAM-capable machines

1. You have gnu/linux or macOS
2. You have at least 64 GB of RAM
3. You have docker engine installed
4. You have an NVIDIA gpu with >=6GB VRAM and supporting CUDA compute compatibility >=7.5

Please, make sure you have fully installed/configure the NVIDIA Container Toolkit checking this guide.

chmod +x build start stop rm

./build

./start

docker container attach text-generation-webui

./stop

./rm

• Currently, GPTQ-for-LLaMa and the monkey patch are not included