Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

overwrite evaluator for particular task #1537

Open
artemorloff opened this issue Mar 6, 2024 · 0 comments
Open

overwrite evaluator for particular task #1537

artemorloff opened this issue Mar 6, 2024 · 0 comments

Comments

@artemorloff
Copy link
Contributor

I have a task that requires context for requests. This leads to changes in evaluator as long as while processing the requests I do not have access to the answers to update the new requests so that they include "the context" - the questions and model answers for the previous ones. Now I want to use the designed evaluator instead of the original one only for this task. Is it possible to somehow overwrite the evaluator through yaml file of the task? Some kind of function that substitutes the original evaluator with my own so that I still can use the original one for other tasks (each task is launched separetely)?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant