Output format of samples has been changed

[christyler3030]

Unless I am missing something this commit changed the format of the saved samples (when log_samples is true.) Previously the samples were saved as jsonlines in a jsonl file but now they are saved as a serialized json list. A fix that maintains the improved serialization of the above commit but also maintains the jsonlines format should be pretty simple; something like:
json_lines = [json.dumps(sample, default=_handle_non_serializable, ensure_ascii=False, ) for sample in samples[task_name]]
filename.write_text(json_lines, encoding="utf-8")

I am happy to implement the fix but I wanted to confirm what the intent was here. Do you want to save as jsonlines or a json list.

[haileyschoelkopf]

Hi! Thanks for reporting this. Indeed Jsonlines is what is desired--if you could implement the fix that would be fantastic!

We'd have to do a quick check to confirm that our Zeno integration and WandB integration which log samples are not broken by this behavior though.