Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Align request path with openai sdk #56

Open
wxpjimmy opened this issue Feb 26, 2024 · 2 comments
Open

Align request path with openai sdk #56

wxpjimmy opened this issue Feb 26, 2024 · 2 comments

Comments

@wxpjimmy
Copy link

wxpjimmy commented Feb 26, 2024

In our scenario, we want to support customer consuming Mistral endpoint with both Mistral AI sdk and OpenAI sdk, but when testing we found that there's a difference between these two clients.

In OpenAI sdk, if customer provides their own base_url, OpenAI sdk won't make any change to that url and there will be no default "v1" prefix added; only when customer doesn't provide base_url, OpenAI sdk will set the url to a default value "https://api.openai.com/v1", this is the code link: https://github.com/openai/openai-python/blob/main/src/openai/_client.py

While in Mistral AI sdk, it will by default add a "v1" prefix for API calls.

To support Mistral AI sdk, the server had to provide the API with "v1” prefix, this also means that customer needs to use different base_url for these 2 sdk:

  • OpenAI: https://<domain>/v1
  • MistralAI: https://<domain>

Of course, to mitigate this impact, we can do some trick in the server side to support both "v1/<api_path>" and "<api_path>", this will be an extra effort for any org who has the same scenario as us.

We'd like to know whether this is by design and whether it's possible to follow the openai sdk way to remove the default "v1" prefix in the request path?

@wxpjimmy wxpjimmy changed the title Align with openai sdk Align request path with openai sdk Feb 26, 2024
@wxpjimmy
Copy link
Author

wxpjimmy commented Mar 8, 2024

@Bam4d any comments for this?

@wxpjimmy
Copy link
Author

wxpjimmy commented Apr 1, 2024

ping for this......

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant