Get up and running with large language models.
curl -fsSL https://ollama.com/install.sh | sh
The official Ollama Docker image ollama/ollama
is available on Docker Hub.
To run and chat with Llama 3.1:
ollama run llama3.1
Ollama supports a list of models available on ollama.com/library
Here are some example models that can be downloaded:
Model | Parameters | Size | Download |
---|---|---|---|
Llama 3.1 | 8B | 4.7GB | ollama run llama3.1 |
Llama 3.1 | 70B | 40GB | ollama run llama3.1:70b |
Llama 3.1 | 405B | 231GB | ollama run llama3.1:405b |
Phi 3 Mini | 3.8B | 2.3GB | ollama run phi3 |
Phi 3 Medium | 14B | 7.9GB | ollama run phi3:medium |
Gemma 2 | 2B | 1.6GB | ollama run gemma2:2b |
Gemma 2 | 9B | 5.5GB | ollama run gemma2 |
Gemma 2 | 27B | 16GB | ollama run gemma2:27b |
Mistral | 7B | 4.1GB | ollama run mistral |
Moondream 2 | 1.4B | 829MB | ollama run moondream |
Neural Chat | 7B | 4.1GB | ollama run neural-chat |
Starling | 7B | 4.1GB | ollama run starling-lm |
Code Llama | 7B | 3.8GB | ollama run codellama |
Llama 2 Uncensored | 7B | 3.8GB | ollama run llama2-uncensored |
LLaVA | 7B | 4.5GB | ollama run llava |
Solar | 10.7B | 6.1GB | ollama run solar |
Note
You should have at least 8 GB of RAM available to run the 7B models, 16 GB to run the 13B models, and 32 GB to run the 33B models.
Ollama supports importing GGUF models in the Modelfile:
-
Create a file named
Modelfile
, with aFROM
instruction with the local filepath to the model you want to import.FROM ./vicuna-33b.Q4_0.gguf
-
Create the model in Ollama
ollama create example -f Modelfile
-
Run the model
ollama run example
See the guide on importing models for more information.
Models from the Ollama library can be customized with a prompt. For example, to customize the llama3.1
model:
ollama pull llama3.1
Create a Modelfile
:
FROM llama3.1
# set the temperature to 1 [higher is more creative, lower is more coherent]
PARAMETER temperature 1
# set the system message
SYSTEM """
You are Mario from Super Mario Bros. Answer as Mario, the assistant, only.
"""
Next, create and run the model:
ollama create mario -f ./Modelfile
ollama run mario
>>> hi
Hello! It's your friend Mario.
For more examples, see the examples directory. For more information on working with a Modelfile, see the Modelfile documentation.
ollama create
is used to create a model from a Modelfile.
ollama create mymodel -f ./Modelfile
ollama pull llama3.1
This command can also be used to update a local model. Only the diff will be pulled.