Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Refactor SASEvaluator #6998

Merged
merged 2 commits into from
Feb 15, 2024
Merged

refactor: Refactor SASEvaluator #6998

merged 2 commits into from
Feb 15, 2024

Conversation

silvanocerza
Copy link
Contributor

Related Issues

Proposed Changes:

Remove preprocessing in SASEvaluator as we're going to use the TextCleaner component from #6997.

Add warm_up() method to cache the model used for evaluation.

Move labels from __init__ to run.

How did you test it?

I updated unit and integration tests.

Notes for the reviewer

This stems from feedback received in PR #6980.

Checklist

@silvanocerza silvanocerza added the ignore-for-release-notes PRs with this flag won't be included in the release notes. label Feb 15, 2024
@silvanocerza silvanocerza self-assigned this Feb 15, 2024
@silvanocerza silvanocerza requested a review from a team as a code owner February 15, 2024 10:36
@silvanocerza silvanocerza requested review from davidsbatista and removed request for a team and davidsbatista February 15, 2024 10:36
@github-actions github-actions bot added topic:tests 2.x Related to Haystack v2.0 type:documentation Improvements on the docs labels Feb 15, 2024
@silvanocerza silvanocerza changed the title feat: Refactor SASEvaluator refactor: Refactor SASEvaluator Feb 15, 2024
@coveralls
Copy link
Collaborator

coveralls commented Feb 15, 2024

Pull Request Test Coverage Report for Build 7917552666

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 17 unchanged lines in 1 file lost coverage.
  • Overall coverage increased (+0.2%) to 89.184%

Files with Coverage Reduction New Missed Lines %
components/eval/sas_evaluator.py 17 63.16%
Totals Coverage Status
Change from base Build 7904082418: 0.2%
Covered Lines: 4939
Relevant Lines: 5538

💛 - Coveralls

haystack/components/eval/sas_evaluator.py Outdated Show resolved Hide resolved
haystack/components/eval/sas_evaluator.py Show resolved Hide resolved
Comment on lines +107 to +109
if not self._similarity_model:
msg = "The model has not been initialized. Call warm_up() before running the evaluator."
raise RuntimeError(msg)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we aren't very consistent with whether the component should call warm_up in run if it hasn't been called already. I'll open an issue to address this separately.

haystack/components/eval/sas_evaluator.py Outdated Show resolved Hide resolved
@silvanocerza silvanocerza merged commit 2a4e6a1 into main Feb 15, 2024
20 checks passed
@silvanocerza silvanocerza deleted the fix-sas-evaluator branch February 15, 2024 15:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.x Related to Haystack v2.0 ignore-for-release-notes PRs with this flag won't be included in the release notes. topic:tests type:documentation Improvements on the docs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants