You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It looks like the issue is here. Each time we create a new variant, we call create_ready_to_use_evaluators which creates these evaluators. However this should only be done when we create an app and now when we create a variant. The solution to call create_ready_to_use_evaluators when we create the app and not here and here. So it should happen here, hope this helps.
To reproduce:
Create an app from cli (agenta init)
Serve variant 1
Create new code (app2.py) and serve variant2
A new exact match and contains json evaluator will be created: https://share.cleanshot.com/QWVWKpFJ
Correct behavior:
From SyncLinear.com | AGE-433
The text was updated successfully, but these errors were encountered: