A collection of exquisitely crafted helm charts for LLMs:
- vLLM
- Text Generation Inference
- Lingo
helm repo add substratusai https://substratusai.github.io/helm
helm repo updateBasic usage:
# Note by default the resource limit is set to 1 GPU
helm install mistral-7b-instruct substratusai/vllm \
--set model=mistralai/Mistral-7B-Instruct-v0.1For Advanced usage see: vLLM Chart Guide