Add OpenELM

[3Samourai]

Apple released several open source LLMs that are designed to run on-device.
Huggingface Link

[thinkverse]

Not supported in llama.cpp yet, there's an issue for it, ggerganov/llama.cpp#6868, labeled as a good first issue, if someone with C++ and Python experience wants to tackle it. 👍

[mertbozkir]

Interesting, I haven't seen this issue, and was trying to upload this model. 🫣

[olumolu]

What is the requirement for llama.cpp.
As far as i see ollama does not depend on llama.cpp

[olumolu]

What is the requirement for llama.cpp.
As far as i see ollama does not depend on llama.cpp

[thinkverse]

As far as i see ollama does not depend on llama.cpp

Ollama makes heavy use of llama.ccp, it's the backend Ollama uses. When you start Ollama, it starts a llama.cpp server. When you chat with an LLM using Ollama is forwards it to the llama.cpp server.

ollama/llm/server.go

Line 73 in 2bed629

func NewLlamaServer(gpus gpu.GpuInfoList, model string, ggml *GGML, adapters, projectors []string, opts api.Options) (LlamaServer, error) {

ollama/server/routes.go

Lines 1315 to 1320 in 2bed629

	if err := runner.llama.Completion(c.Request.Context(), llm.CompletionRequest{
	Prompt: prompt,
	Format: req.Format,
	Images: images,
	Options: opts,
	}, fn); err != nil {

You can see the llama.cpp submodude under https://github.com/ollama/ollama/tree/main/llm.

[olumolu]

ggerganov/llama.cpp#6960

[thinkverse]

Update on the OpenELM support, a draft PR has been opened ggerganov/llama.cpp#6986, and @joshcarp is looking for anyone to help out. I'm certain it would be appreciated if anyone who has experience with C++, Python, or something related can help. 👍

[UmutAlihan]

looking very forward to be able to infer using ollama with these model ggufs: https://huggingface.co/apple/OpenELM

[olumolu]

ggerganov/llama.cpp#7359
Now this has been merged so this model can now easily featured in ollama.

[thinkverse]

this model can now easily featured in ollama

Ollama needs to update its version of llama.cpp first, maybe #5475 could be updated to include the OpenELM PR, @jmorganca?

[CamJN]

llama.cpp d7fd29f adds OpenELM support

the latest release of ollama is commit e4ff732, the llama.cpp submodule is pinned to a8db2a9 for that commit

a8db2a9 is after and contains the changes of d7fd29f as can be seen here: https://github.com/ggerganov/llama.cpp/commits/a8db2a9ce64cd4417f6a312ab61858f17f0f8584/

So adding OpenELM support should be possible now.

[olumolu]

What is the status of this when i can assume to have support for this.?

[tomasmcm]

I've uploaded the 3b-intruct-q5_K_M version here https://ollama.com/tomasmcm/openelm
But it does not seem these models are that great. Unless you ask something simple like Can you translate this to french? "Hi there, how are you doing?" it hallucinates a lot.

[JeremyLaurenson]

I think they are really expecting people to fine-tune this for their particular application and so basic sentence understanding is about it