-
Notifications
You must be signed in to change notification settings - Fork 124
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
InferenceEndpoint Generator fails for AWS because of missing Authorization header #722
Comments
Thanks, this is useful to know. We'd like to fix it. For clarity, I assume this is about @erikinfo Do you know how we can get a sample endpoint for testing & validation? @jmartin-tech I think an approach similar to how |
Yep, Unfortunately, I think there is no way besides hosting one yourself. I could also try and help with testing.
Please note that this method should just be an helper to pin point to the headers attributes that are needed. The headers attribute should therefore be updated in the huggingface.InferenceEndpoint. Please allow the following header attributes to be added:
|
I think the
|
@erikinfo Thanks tons for this detailed example, it should be really helpful. Also thanks for volunteering to help test. I hope we can get started using this guide, https://huggingface.co/docs/sagemaker/inference @jmartin-tech This could work. Two separate generator classes for one named product (https://huggingface.co/docs/inference-endpoints/index) seems a little unintuitive to me, but if it reduced tech debt in exchange for a reasonably-sized dependency, that's a win. Maybe something for or closely after the |
The Authorization request header is necessary to establish a connection the the endpoint hosted on AWS Sagemaker.
The InferenceEndpoint inside huggingface.py should therefore have a new field consisting of auth header sent with the request, similiar to how API keys were treated using environment variables.
The text was updated successfully, but these errors were encountered: