Figure out what to do about gpt-4-turbo-preview and gpt-4-0125-preview

[simonw]

Today OpenAI they made some changes - https://openai.com/blog/new-embedding-models-and-api-updates - including:

Today, we are releasing an updated GPT-4 Turbo preview model, gpt-4-0125-preview. This model completes tasks like code generation more thoroughly than the previous preview model and is intended to reduce cases of "laziness" where the model doesn't complete a task. The new model also includes the fix for the bug impacting non-English UTF-8 generations.

For those who want to be automatically upgraded to new GPT-4 Turbo preview versions, we are also introducing a new gpt-4-turbo-preview model name alias, which will always point to our latest GPT-4 Turbo preview model.

See also https://platform.openai.com/docs/models/gpt-4-and-gpt-4-turbo

Problem is the current code does this:

llm/llm/default_plugins/openai_models.py

Line 34 in bf082cb

register(Chat("gpt-4-1106-preview"), aliases=("gpt-4-turbo", "4-turbo", "4t"))

So -m gpt-4-turbo now gets you that slightly older model, not the new one that's less lazy!

[simonw]

Even though it's technically a breaking change, I think the right thing to do is to use that default gpt-4-turbo alias for gpt-4-turbo-preview instead. That seems a better fit to how users would expect these models to work.

I'll document this in the release notes and ensure -m gpt-4-1106-preview works going forward.

[simonw]

Another thing that will be a bit less disruptive once llm-openai is split out into a plugin I can release separately.

[simonw]

Got these working:

$ llm -m gpt-4-1106-preview 'say hi'
Hello! How can I assist you today?
$ llm -m gpt-4-0125-preview 'say hi'
Hi! How can I assist you today?
$ llm -m gpt-4-0124-preview 'say hi'
Error: 'gpt-4-0124-preview' is not a known model
$ llm -m gpt-4-turbo-preview 'say hi'
Hello! How can I assist you today?

[simonw]

Relevant documentation: https://llm.datasette.io/en/latest/openai-models.html#openai-language-models