Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: Fix LLMEvaluator serialization #7818

Merged
merged 2 commits into from
Jun 7, 2024
Merged

fix: Fix LLMEvaluator serialization #7818

merged 2 commits into from
Jun 7, 2024

Conversation

shadeMe
Copy link
Collaborator

@shadeMe shadeMe commented Jun 6, 2024

Related Issues

Proposed Changes:

The inputs init parameter of the LLMEvaluator component accepts a list of tuples containing an input name and its corresponding type. Previously, we were saving this parameter as-is - problematic due to the serialization code supporting neither raw tuples nor raw types. This PR fixes the above issue by converting the tuples to lists and the correctly serializing the types.

How did you test it?

Unit tests

Checklist

@shadeMe shadeMe added type:bug Something isn't working topic:eval 2.x Related to Haystack v2.0 labels Jun 6, 2024
@shadeMe shadeMe marked this pull request as ready for review June 6, 2024 13:14
@shadeMe shadeMe requested review from a team as code owners June 6, 2024 13:14
@shadeMe shadeMe requested review from dfokina and masci and removed request for a team June 6, 2024 13:14
@coveralls
Copy link
Collaborator

coveralls commented Jun 6, 2024 •
edited
Loading

Pull Request Test Coverage Report for Build 9401492718

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 2 unchanged lines in 1 file lost coverage.
  • Overall coverage increased (+0.003%) to 89.738%
Files with Coverage Reduction New Missed Lines %
components/evaluators/llm_evaluator.py 2 95.5%
Totals Coverage Status
Change from base Build 9400937231: 0.003%
Covered Lines: 6777
Relevant Lines: 7552
💛 - Coveralls

@shadeMe shadeMe mentioned this pull request Jun 6, 2024
Merged
@coveralls
Copy link
Collaborator

coveralls commented Jun 6, 2024 •
edited
Loading

Pull Request Test Coverage Report for Build 9401458578

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 2 unchanged lines in 1 file lost coverage.
  • Overall coverage increased (+0.003%) to 89.738%
Files with Coverage Reduction New Missed Lines %
components/evaluators/llm_evaluator.py 2 95.5%
Totals Coverage Status
Change from base Build 9400937231: 0.003%
Covered Lines: 6777
Relevant Lines: 7552
💛 - Coveralls

@coveralls
Copy link
Collaborator

coveralls commented Jun 6, 2024 •
edited
Loading

Pull Request Test Coverage Report for Build 9401565701

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 2 unchanged lines in 1 file lost coverage.
  • Overall coverage increased (+0.003%) to 89.738%
Files with Coverage Reduction New Missed Lines %
components/evaluators/llm_evaluator.py 2 95.5%
Totals Coverage Status
Change from base Build 9400937231: 0.003%
Covered Lines: 6777
Relevant Lines: 7552
💛 - Coveralls

@shadeMe shadeMe requested review from silvanocerza and removed request for masci June 7, 2024 10:08
@github-actions github-actions bot added the type:documentation Improvements on the docs label Jun 7, 2024
@coveralls
Copy link
Collaborator

coveralls commented Jun 7, 2024 •
edited
Loading

Pull Request Test Coverage Report for Build 9415331669

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 2 unchanged lines in 1 file lost coverage.
  • Overall coverage increased (+0.003%) to 89.753%
Files with Coverage Reduction New Missed Lines %
components/evaluators/llm_evaluator.py 2 95.5%
Totals Coverage Status
Change from base Build 9415009179: 0.003%
Covered Lines: 6823
Relevant Lines: 7602
💛 - Coveralls

@shadeMe shadeMe merged commit 63226da into deepset-ai:main Jun 7, 2024
22 checks passed
@shadeMe shadeMe deleted the fix/llmevaluator-serde branch June 7, 2024 10:49
@lbux lbux mentioned this pull request Jun 21, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@silvanocerza silvanocerza silvanocerza approved these changes

@dfokina dfokina Awaiting requested review from dfokina dfokina is a code owner automatically assigned from deepset-ai/documentation

Assignees
No one assigned
Labels
2.x Related to Haystack v2.0 topic:eval topic:tests type:bug Something isn't working type:documentation Improvements on the docs
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Serialization issues with LLMEvaluator and child classes
3 participants
@shadeMe @coveralls @silvanocerza