openai.InternalServerError: the model generated invalid Unicode output

[djstrong]

While testing gpt3.5-instruct I encounter the error for a few tasks:

2024-05-04:13:39:36,070 INFO     [_client.py:1026] HTTP Request: POST https://api.openai.com/v1/completions "HTTP/1.1 429 Too Many Requests"
2024-05-04:13:39:36,070 INFO     [_base_client.py:952] Retrying request to /completions in 1.000000 seconds
2024-05-04:13:39:37,453 INFO     [_client.py:1026] HTTP Request: POST https://api.openai.com/v1/completions "HTTP/1.1 200 OK"
 99%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▊ | 490/494 [09:08<00:05,  1.41s/it]
2024-05-04:13:39:37,933 INFO     [_client.py:1026] HTTP Request: POST https://api.openai.com/v1/completions "HTTP/1.1 500 Internal Server Error"
2024-05-04:13:39:37,933 INFO     [_base_client.py:952] Retrying request to /completions in 0.788895 seconds
2024-05-04:13:39:38,906 INFO     [_client.py:1026] HTTP Request: POST https://api.openai.com/v1/completions "HTTP/1.1 429 Too Many Requests"
2024-05-04:13:39:38,906 INFO     [_base_client.py:952] Retrying request to /completions in 1.000000 seconds
2024-05-04:13:39:40,275 INFO     [_client.py:1026] HTTP Request: POST https://api.openai.com/v1/completions "HTTP/1.1 500 Internal Server Error"
Traceback (most recent call last):
  File "~/lm-evaluation-harness/lm_eval/models/utils.py", line 333, in wrapper
    return func(*args, **kwargs)
  File "~/lm-evaluation-harness/lm_eval/models/openai_completions.py", line 72, in completion
    return client.completions.create(**kwargs)
  File "~/venv/lib/python3.10/site-packages/openai/_utils/_utils.py", line 303, in wrapper
    return func(*args, **kwargs)
  File "~/venv/lib/python3.10/site-packages/openai/resources/completions.py", line 559, in create
    return self._post(
  File "~/venv/lib/python3.10/site-packages/openai/_base_client.py", line 1088, in post
    return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
  File "~/venv/lib/python3.10/site-packages/openai/_base_client.py", line 853, in request
    return self._request(
  File "~/venv/lib/python3.10/site-packages/openai/_base_client.py", line 916, in _request
    return self._retry_request(
  File "~/venv/lib/python3.10/site-packages/openai/_base_client.py", line 958, in _retry_request
    return self._request(
  File "~/venv/lib/python3.10/site-packages/openai/_base_client.py", line 916, in _request
    return self._retry_request(
  File "~/venv/lib/python3.10/site-packages/openai/_base_client.py", line 958, in _retry_request
    return self._request(
  File "~/venv/lib/python3.10/site-packages/openai/_base_client.py", line 930, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.InternalServerError: Error code: 500 - {'error': {'message': 'Failed to create completion as the model generated invalid Unicode output. Unfortunately, this can happen in rare situations. Consider reviewing your prompt or reducing the temperature of your request. You can retry your request, or contact us through our help center at help.openai.com if the error persists. (Please include the request ID req_b699d9b8647f705cefa370c62e0bddad in your message.)', 'type': 'server_error', 'param': None, 'code': 'invalid_model_output'}}
2024-05-04:13:39:43,711 INFO     [_client.py:1026] HTTP Request: POST https://api.openai.com/v1/completions "HTTP/1.1 500 Internal Server Error"
2024-05-04:13:39:43,711 INFO     [_base_client.py:952] Retrying request to /completions in 0.810511 seconds
2024-05-04:13:39:44,880 INFO     [_client.py:1026] HTTP Request: POST https://api.openai.com/v1/completions "HTTP/1.1 500 Internal Server Error"
2024-05-04:13:39:44,880 INFO     [_base_client.py:952] Retrying request to /completions in 1.789714 seconds
2024-05-04:13:39:47,024 INFO     [_client.py:1026] HTTP Request: POST https://api.openai.com/v1/completions "HTTP/1.1 500 Internal Server Error"

I would like to treat this error as a wrong answer of the model, but now it retries infinite number of times.