Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Docs] Check if ChatGPT api on azure needs docs updates #4694

Closed
agnieszka-m opened this issue Apr 18, 2023 · 4 comments
Closed

[Docs] Check if ChatGPT api on azure needs docs updates #4694

agnieszka-m opened this issue Apr 18, 2023 · 4 comments
Assignees
Labels
type:documentation Improvements on the docs

Comments

@agnieszka-m
Copy link
Contributor

No description provided.

@agnieszka-m
Copy link
Contributor Author

#4675 (review)

@agnieszka-m agnieszka-m added the type:documentation Improvements on the docs label Apr 18, 2023
@agnieszka-m
Copy link
Contributor Author

Check with Bogdan if the PromptNode doc https://docs.haystack.deepset.ai/docs/prompt_node#models needs updates because of the new invocation layer.

@recrudesce
Copy link
Contributor

recrudesce commented Apr 18, 2023

I think the only thing that'll need to be updated is the fact that Azure call their model gpt-35-turbo and not gpt-3.5-turbo - I believe the GPT-4 models are named the same as OpenAI. Everything else is as per the "if you're using Azure, send these 2 extra variables" that's already mentioned here: https://docs.haystack.deepset.ai/docs/prompt_node#using-azure-openai-service

@dfokina dfokina closed this as completed Apr 19, 2023
@recrudesce
Copy link
Contributor

Sorry, sorry sorry... also you need to make sure the API version is either not set (as the integration will set it correctly), or set to 2023-03-15-preview

In fact, you don't even need to set the API version in the base Azure OpenAI request either, because it's set by default in the following: https://github.com/deepset-ai/haystack/blob/main/haystack/nodes/prompt/invocation_layer/azure_open_ai.py#L20

Worth just removing the api_version key/value from the model_kwargs in the docs, as there's no real reason to override it generally:

prompt_azure_open_ai = PromptModel(
    model_name_or_path="text-davinci-003", 
    api_key="<your-azure-openai-key>", 
    model_kwargs={
        "api_version": "2022-12-01",
        "azure_base_url":"https://<your-endpoint>.openai.azure.com", 
        "azure_deployment_name": "<your-deployment-name>",
    }
)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type:documentation Improvements on the docs
Projects
None yet
Development

No branches or pull requests

3 participants