ignore_case option for the Includes eval

[trizko]

I have been creating a new eval using the Includes template. I was surprised to see that there is no option to ignore the casing and so I added it directly to the class and tried it out. It works fine and I would like to make a pull request to add this argument. It would be used in the registry yaml file like so:

spanish_english_idioms:
  id: spanish_english_idioms.test.v1
  metrics: [accuracy]
spanish_english_idioms.test.v1:
  class: evals.elsuite.basic.includes:Includes
  args:
    samples_jsonl: spanish_english_idioms/samples.jsonl
    ignore_case: true

Would anyone be interested in this? I feel like it would be helpful to have as an option for folks that do not want to or know how to code. If this functionality already exists somewhere in the repo, let me know and I'll close this issue out.

[jwang47]

Sounds useful, feel free to open a PR to add this.

[trizko]

Here is the link to the PR #655

[tgf9]

Thanks, @trizko! This would be helpful to me as well! 🙏

[jwang47]

Closing since this has been fixed. Thanks @trizko!