Ollama support

[ThatOneCalculator]

Support for self-hosted Ollama servers would be great

[Taewan-P]

Do they have a unified output format? Any API references?

[ThatOneCalculator]

https://github.com/ollama/ollama/blob/main/docs%2Fapi.md

[randoentity]

Wouldn't it make more sense to just use the openai api spec? That adds support for many inference servers, including ollama (didn't they also just use the openai api spec?).

[Taewan-P]

@randoentity Wow, after seeing your comment I tried searching for that... and surprisingly it is (link)
This will make it more easy to implement the feature. Thanks! 😃

[ThatOneCalculator]

I would recommend against using the OpenAI API layer, as it's still WIP and missing many features.

[Taewan-P]

There is not really much feature we need rn. If chat completion works, I think that's good for now.

[ThatOneCalculator]

Ok, the chat completion works so as long as remote models can be listed with the regular Ollama API, I don't see an issue. It would be nice to have vision/etc in the future though