Default behaviors not respected?

[rdegges]

Hi there! First off, thanks for building this project! It's super cool and I'm using it extensively <3

I'm planning to roll this out to some colleagues at my organization and ran into an issue. In my case we have a few different environment variables provided with API credentials so that we have a number of models users can access.

I've set the environment variable NEXT_PUBLIC_DEFAULT_MODEL to the specified model I'd like to use as the default (gpt-4-32k), but the setting doesn't appear to be respected. The same is true for NEXT_PUBLIC_DEFAULT_TEMPERATURE when set to 1, which is the default value ChatGPT uses and what I'd like to default temperature to for my users.

Any ideas what I may be doing incorrectly here? Thanks again! <3

[jorge-menjivar]

I am currently working on this branch: https://github.com/jorge-menjivar/unSAGED/tree/model-params-upgrade
Just keep in mind that this update will require a quick change to the database, in order to upgrade from an older version. See the README for more details.

This update will change some things:

1. The names of the env variables will change, allowing for one default var per AI vendor (OpenAI, Google, Ollama, etc).
2. It includes support for other parameters like top_p, seed, etc.
3. Not setting a default var won't force a temperature like 0.7 anymore. It will simply leave them blank in the completion request.

The latest commit should also fix the issue with the default model not being respected.

[jorge-menjivar]

Also if the default model env variable is not set, new conversations will default to the model of the last conversation that was used.

[rdegges]

Oh thank you! This is awesome.