Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FaithfulnesEvaluator and ContextRelevanceEvaluator cannot be loaded via Pipeline.from_dict #7884

Open
1 task
tstadel opened this issue Jun 18, 2024 · 0 comments

Comments

@tstadel
Copy link
Member

tstadel commented Jun 18, 2024

Describe the bug
As FaithfulnesEvaluator and ContextRelevanceEvaluator do not have the component decorator, Pipeline.from_dict cannot resolve them. The super-init workaround decribed in #7824 should do.

Error message
haystack.core.errors.PipelineError: Component 'haystack.components.evaluators.faithfulness.FaithfulnessEvaluator' not imported.

Expected behavior
They can be loaded from_dict

Additional context
Add any other context about the problem here, like document types / preprocessing steps / settings of reader etc.

To Reproduce

import yaml
from haystack import Pipeline
pipeline_yaml = """
    components:
        generator:
            init_parameters: {}
            type: haystack.components.evaluators.faithfulness.FaithfulnessEvaluator
       """
pipeline_config = yaml.load(pipeline_yaml, Loader=yaml.FullLoader)
pipeline = Pipeline.from_dict(pipeline_config)

FAQ Check

System:

  • OS:
  • GPU/CPU:
  • Haystack version (commit or version number):
  • DocumentStore:
  • Reader:
  • Retriever:
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant