• Allow SendToOllamaAsync() to be overridden #87 to support streaming responses in Blazor WASM #84
• Update RequestOptions #88 with new properties MainGpu, UseMmap, LowVram and more.

• Exposed RequestOptions (to set the temperature, etc.) when using the Chat class #77

• Added support for default request headers to the OllamaApiClient and custom request headers to requests against the Ollama API.

• Added OllamaApiClient.IncomingJsonSerializerOptions to control how responses from Ollama are deserialized #78

• Removed the OllamaApiConsole demo which is now located in an own repository to reduce package updates
• Added OllamaApiClient.OutgoingJsonSerializerOptions to control how requests to Ollama are serialized #75

Improved tool chat demos

Fixed tool chat demo and added more unit tests

As announced in discussion #63, this release went through major code changes (see pull request #64) and drops a lot of unnecessary code.

OllamaSharp now uses the IAsyncEnumerable syntax. The following paragraph shows the breaking changes from version 2 to 3.

Breaking changes

Methods on IOllamaApiClient

• Chat() now returns IAsyncEnumerable<ChatResponseStream?> (prefer the Chat class to build interactive chats)
• SendChat()/StreamChat() (sync/async) were removed in favor for the new Chat() method
• CreateModel() overload accepting streaming callbacks was removed, use the IAsyncEnumerable<> syntax
• PullModel() overload accepting streaming callbacks was removed, use the IAsyncEnumerable<> syntax
• PushModel() overload accepting streaming callbacks was removed, use the IAsyncEnumerable<> syntax
• GenerateEmbeddings() is now Embed() to follow the Ollama API naming
• ShowModelInformation() is now ShowModel() to follow the Ollama API naming
• StreamCompletion() is now Generate() to follow the Ollama API naming
• GetCompletion() (sync) was removed in favor for Generate() method

Chat class

• Send() & SendAs() overloads accepting streaming callbacks were removed, use the IAsyncEnumerable<> syntax
• Constructor cannot use streamer callbacks anymore, as Send() & SendAs()stream directly

Classes

• GenerateEmbeddingRequest → EmbedRequest
• GenerateEmbeddingResponse → EmbedResponse
• GenerateCompletionRequest → GenerateRequest
• GenerateCompletionResponseStream → GenerateResponseStream
• GenerateCompletionDoneResponseStream → GenerateDoneResponseStream
• ChatResponse was removed as it was only used by the old signature of the Chat() method

Streaming callbacks

A special note to the streaming callbacks that have been removed. Before version 3, it was possible to something like this:

var answer = await chat.Send("hi!", r => Console.Write(r.Message));

Since version 3, all the method overloads accepting these streaming callbacks have been removed in favor for the IAsyncEnumerable syntax:

await foreach (var answerToken in chat.Send("hi!"))
    Console.Write(answerToken);

The second approach, the IAsyncEnumerable syntax, is the modern one and easier to read. However, one tiny detail is missing: You cannot stream the responses and get the whole answer as result at the same time anymore.

Theoretically, you would have to do something like this if you wanted to get the whole result value:

var builder = new StringBuilder();
await foreach (var answerToken in chat.Send("hi!"))
    builder.Append(answerToken);
Console.WriteLine(builder.ToString());

StreamToEnd()

To make this easier, OllamaSharp provides an extension method for IAsyncEnumerable that streams its items to a single result value: StreamToEnd().

With this, you can simply stream the responses into a single value:

var answer = await chat.Send("hi!").StreamToEnd();

If you still want to do something with each item in the IAsyncEnumerable, you can even pass a item callback to the method:

var answer = await chat.Send("hi!").StreamToEnd(token => { ... });

This way, the OllamaApiClient can stream its results as IAsyncEnumerable and the stream processing is done in an extension method instead of having dozens confusing method overloads.