-
Notifications
You must be signed in to change notification settings - Fork 2.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Evaluating LLMs on QA Tasks #65
Comments
Hey there! Thanks for sharing your idea on how to evaluate an LLM on various question-answering tasks. I really appreciate your contribution and I think your pseudocode provides a great starting point for exploring and understanding the evaluation process. And you're right, there's always room for improvement, so I encourage you and others to share your thoughts and experiences to help enhance the understanding and implementation of this process. Keep up the good work! |
quite impressive |
@slavakurilyak ok I know this is a weird question, but...did you generate this with ChatGPT? 👀 It has a very similar tone. The pseudocode, the disclaimers, the step-by-step thing. It's very similar to when I ask ChatGPT for coding help. |
Maybe
…On Wed, 15 Mar, 2023, 11:07 am ricky-sb, ***@***.***> wrote:
@slavakurilyak <https://github.com/slavakurilyak> ok I know this is a
weird question, but...did you generate this with ChatGPT? 👀
—
Reply to this email directly, view it on GitHub
<#65 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AY2ADJTJH6UB7EAO3CU2WRTW4FIQLANCNFSM6AAAAAAV3CYJAM>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Id like to contribute. |
🤓 Me ✌️ from 🇷🇼 |
The tasks described:
should already be supported by Evals, as you can make the input a Chat conversation object up until the next turn (which is when the model would respond) |
Here's an idea on how to evaluate an LLM on various question-answering tasks, such as open-domain question answering, conversational question answering, answer selection, community question answering, and knowledge base question answering:
I'd like to add a caveat about the pseudocode I provided:
The text was updated successfully, but these errors were encountered: