TimeOut on long queries

[Neo-vortex]

if queries are long enough the client will timeout before the actual result is retrieved:

System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
 ---> System.TimeoutException: The operation was canceled.
 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
 ---> System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
 ---> System.Net.Sockets.SocketException (125): Operation canceled
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   at OllamaApiClient.GenerateCompletion(GenerateCompletionRequest generateRequest, IResponseStreamer`1 streamer)
   at OllamaApiClient.GetCompletion(String prompt, String model, ConversationContext context)
   at llamaAssistance.AiResponseGenerator.GenerateResponse(String query) in /home/neo/Projects/Personal/masstransit/llamaAssistance/llamaAssistance/AiResponseGenerator.cs:line 17
   at Program.<Main>$(String[] args) in /home/neo/Projects/Personal/masstransit/llamaAssistance/llamaAssistance/Program.cs:line 9
   at Program.<Main>(String[] args)

[awaescher]

What exactly do you mean by that?

[Neo-vortex]

ask something that takes a long time to generate response (more than 100 sec) the client will timeout

[awaescher]

If the connection cannot be established, the app should be able to recognize and report that.
I highly recommend using Polly which works with the native HttpClient OllamaSharp is using internally.

[Neo-vortex]

Why do you close issues so quickly!
If the connection can not be established a connection refuse will trigger almost immediately and has nothing to do with request timeouts.

[awaescher]

Timeouts can always happen if there's network in between, having apps freezing endlessly is no option.