-
Notifications
You must be signed in to change notification settings - Fork 271
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] Published API Token limit #439
Comments
No sorry, it's a different issue. I can send the exact message I want to post to an API in a chatbot through the application, while it will fail while using the API. Surely it should be possible to utilise the same logic for published API and API to post a chat message? |
Sorry, I misunderstood. Yes surely they utilize the same logic but there is a implementation difference. Published api uses chat method and the other uses process_chat_input. If the same message invokes error is posted on the post a chat interface, it would make the same err. |
It does not generate the same error when you post the same message to the API and the chat interface, see below.
When posting to the Chat interface, it works, see my screenshot. I've included the text that I used to test this as a file. (I let Claude generate a long text) This is the code I use to post to the API:
Please do keep in mind that if you were to test this yourself, the API call will return a success, but if you go to your Cloudwatch SQS logs, you will see that there was an error. It might also be in the way I post to the API? Would love to hear a way to make this work since we would like to work with published API's and longer texts. |
Thank you for the detail. The log shows that the bot looks like using Knowledge a.k.a. RAG but your chat screen does not look like using Knowledge. Could you describe the procedure to reproduce concretely and precisely as much as possible? |
I did paste my code and procedure in my previous post, which is how I call the API from a published bot API. Also for clarification, both ways I showed are using a bot with no knowledge - only a custom instruction. (Editting in your own published bot API and API Key) So again: 2048+ characters text message -> Chat interface -> Bot with no knowledge (Custom instructions only) -> Response |
Actually got an update: Using a bot with knowledge generates the same error as calling a POST to the API of a bot without knowledge.
So when posting a long text to a bot without knowledge, it works, but posting it to a bot with knowledge - it does not work. Is the cause of this known? Any workaround present/is it on the roadmap? |
Describe the bug
When doing a POST to a published API with a large message, we run into this error:
[ERROR] ValidationException: An error occurred (ValidationException) when calling the InvokeModel operation: Malformed input request: #/texts/0: expected maxLength: 2048, actual: 8575, please reformat your input and try again.
To Reproduce
API POST to a published API with a message larger than 2048 tokens.
Screenshots
Additional context
Is there any solution to this or workaround? For example I could split up the message in different chunks - however I would like to send it all as 1 message so that I get 1 response to the full message.
The text was updated successfully, but these errors were encountered: