[AGE-276] Improve robustness AI critique evaluation

[mmabrouk]

Problem Description:
There are cases where the AI critique evaluator provides a response that includes both a numerical rating and a tex, such as "8 - The output is accurate, well-written, and concise, but lacks detailed information." Currently this fails the evaluation.

Proposed Solution:

• Improve the default AI critique prompt to minimize such instances.
• Use regular expressions to extract numerical values from the evaluator's responses.
• Issue: Evaluation Should Not Fail If Any Individual Evaluator Case Fails #1234

_AGE-276

[mmabrouk]

@jp it turns out this is a duplicate issue. I have added some comments (the ones mentioned this morning in AGE-370)

[matallanas]

I was looking to the code and that is mainly that gpt3.5-turbo has a lot of errors. Could also be configurable the model to use as evaluator. In addition, gpt4o-mini is cheaper now than gpt3.5. The example prompt can be more exact also to match the output required.

[aakrem]

I think a better solution would be to add the model to be used while configuring the ai-ciritique evaluator.

[mmabrouk]

I agree that it makes sense to provide a model choice for the user together with a way to configure the evaluator (@jp-agenta it might make sense to think how we can have the same logic for configuring RAG evaluators and AI Critic). However, we still need to provide good defaults.

First solution is to lower the temperature. Second is to ask for json.

[matallanas]

To test my hipothesis I made a webhook with the llm judge and gpt-3.5-turbo. With a temperature of 0.2 and teh following prompt

We have an LLM App that we want to evaluate its outputs. Based on the prompt and the parameters provided below evaluate the output based on the evaluation strategy below:
Evaluation strategy: 0 to 1, 0 is very bad and 1 is very good.
Expected Answer Column:{correct_answer}
Evaluate this: {output}

Answer ONLY with the floating number nothing else.

90% of the cases get back only the number of the answer.

[mmabrouk]

@matallanas I have created a PR based on this discussion: #1938