Timeouts with mistral-large-latest

[asolanki]

Problems with testing identical prompt across different Mistral models, and having consistent issue with mistral-large-latest.

Prompt length as calculated by GPT tokenizer: 2,196 tokens and 11,331 characters

Note: this exact prompt works consistently with mistral-medium-2312 and mistral-medium-latest

So far, I have tried increasing timeout on client instantiation to ~5 minutes to see if that helped (though medium works fine on default).

I have tried both synchronous and streaming connections. Here is my current situation:

Error:

Unexpected exception (RemoteProtocolError): peer closed connection without sending complete message body (incomplete chunked read)

relevant code snippet:

stream_response = client.chat_stream(model=model, messages=messages, temperature=TEMPERATURE)
for chunk in stream_response:
  print(chunk)
  summary = summary + chunk.choices[0].delta.content