Docker image for the Text Generation Web UI: A Gradio web UI for Large Language Models. Supports transformers, GPTQ, llama.cpp (GGUF), Llama models
Note
The legacy APIs no longer work with the latest version of the
Text Generation Web UI. They were deprecated in
November 2023 and have now been completely removed.
If you want to use the LEGACY APIs, please set the image tag
to 1.9.5. You will also have to add port 6000 for the
legacy REST API and/or port 6005 for the legacy Websockets API.
- Ubuntu 22.04 LTS
- CUDA 12.1.1
- Python 3.10.12
- Text Generation Web UI
- Torch 2.1.2
- xformers 0.0.23.post1
- runpodctl
- croc
- rclone
- speedtest-cli
- screen
- tmux
This image is designed to work on RunPod. You can use my custom RunPod template to launch it on RunPod.
docker run -d \
--gpus all \
-v /workspace \
-p 3000:3001 \
-p 5000:5001 \
-p 8888:8888 \
-e JUPYTER_PASSWORD=Jup1t3R! \
ashleykza/oobabooga:latestYou can obviously substitute the image name and tag with your own.
Pull requests and issues on GitHub are welcome. Bug fixes and new features are encouraged.
You can contact me and get help with deploying your container to RunPod on the RunPod Discord Server below, my username is ashleyk.
