-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: improve context relevancy metric #7964
Conversation
Pull Request Test Coverage Report for Build 9761382356Details
💛 - Coveralls |
Pull Request Test Coverage Report for Build 9763512100Details
💛 - Coveralls |
Pull Request Test Coverage Report for Build 9763863648Details
💛 - Coveralls |
Pull Request Test Coverage Report for Build 9764606380Details
💛 - Coveralls |
Pull Request Test Coverage Report for Build 9764679440Details
💛 - Coveralls |
Pull Request Test Coverage Report for Build 9796828476Warning: This coverage report may be inaccurate.This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.
Details
💛 - Coveralls |
Pull Request Test Coverage Report for Build 9807039259Warning: This coverage report may be inaccurate.This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.
Details
💛 - Coveralls |
Pull Request Test Coverage Report for Build 9807972372Warning: This coverage report may be inaccurate.This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.
Details
💛 - Coveralls |
NOTE: blocked until the 2.4 release |
Pull Request Test Coverage Report for Build 10041187329Details
💛 - Coveralls |
Co-authored-by: Madeesh Kannan <[email protected]>
Co-authored-by: Madeesh Kannan <[email protected]>
@shadeMe thanks for the feedback - I've updated the PR |
Let's wait for @anakin87's review before merging. |
Co-authored-by: Stefano Fiorucci <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Feel free to merge once the following line is updated
"outputs" must be a dictionary with "statements" and "statement_scores". |
Related Issues
Proposed Changes:
The
ContextRelevanceEvaluator
now returns a list of relevant sentences for each context, instead of all the sentences in a context. Also, a score of 1 is now returned if a relevant sentence is found, and 0 otherwise.How did you test it?
Notes for the reviewer
Checklist
fix:
,feat:
,build:
,chore:
,ci:
,docs:
,style:
,refactor:
,perf:
,test:
.