-
Notifications
You must be signed in to change notification settings - Fork 93
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Same output for any system prompt #103
Comments
Hi @yangzhj53, Thank you for your interest in our work. It looks like the model is not respecting the system/user prompt in this particular case which can be considered as one of the limitations of the model. May be, using some language only instruction data (e.g. ShareGPT data) along with the VideoInstruct-100K improves the performance. Please do share if you fix this trend. Thank you and good luck :) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I followed the steps in the tutorial to evaluate the results of the model on MSVD-QA, but I found that regardless of how my system prompt was set, even if the model was asked to answer 'Yes', its answer was almost consistent. Is this reasonable?
The text was updated successfully, but these errors were encountered: