Docker image for the Text Generation Web UI: A Gradio web UI for Large Language Models. Supports transformers, GPTQ, llama.cpp (GGUF), Llama models
NOTE: If you want to use the LEGACY API with chat support, please set the image tag to 1.9.5. The legacy chat API no longer works from version 1.10.0 onwards.
- Ubuntu 22.04 LTS
- CUDA 12.1.1
- Python 3.10.12
- Text Generation Web UI
- Torch 2.1.2
- xformers 0.0.23.post1
- runpodctl
- croc
- rclone
This image is designed to work on RunPod. You can use my custom RunPod template to launch it on RunPod.
docker run -d \
--gpus all \
-v /workspace \
-p 3000:3001 \
-p 5000:5001 \
-p 6000:6050 \
-p 6005:6055 \
-p 8888:8888 \
-e JUPYTER_PASSWORD=Jup1t3R! \
ashleykza/oobabooga:latest
You can obviously substitute the image name and tag with your own.
Pull requests and issues on GitHub are welcome. Bug fixes and new features are encouraged.
You can contact me and get help with deploying your container to RunPod on the RunPod Discord Server below, my username is ashleyk.