-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Extra generations even if all field names are in the completion #1108
Comments
So the change is sometimes you want a field in the completion without any content? |
So... the correct fix looks like below:
@mikeedjones - does above look good to you? If yes, I can submit a PR |
do you want the model to fill hint? |
@mikeedjones No. Its an input field that can be empty |
My proposed fix above is not enough. When there is a legitimate extend generations scenario but there is an empty input field, the code still gets into a loop and ends with "Max depth exceeded - failed to complete in one pass - increase max_tokens". The function "get_all_fields_following_missing_field" should really be implemented as "get_all_output_fields_following_missing_output_field". How to distinguish input vs output fields given an instance of the Example class?? Seems to me you need to pass the Signature instance as an argument all the way down The work-around for now is to never leave an input field empty. |
Testing with Llama-3 70B, was seeing duplicate generations.
I traced it to this code in predict.py:
Not exactly sure why the "if" check fails incorrectly.
I fixed it by doing this:
The text was updated successfully, but these errors were encountered: