Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dataframes of EvalResult are not the same after (de)serializing #4905

Closed
1 task done
tstadel opened this issue May 12, 2023 · 0 comments · Fixed by #4906
Closed
1 task done

Dataframes of EvalResult are not the same after (de)serializing #4905

tstadel opened this issue May 12, 2023 · 0 comments · Fixed by #4906
Assignees

Comments

@tstadel
Copy link
Member

tstadel commented May 12, 2023

Describe the bug
After calling EvaluationResult.save() and EvaluationResult.load() the underlying dataframes are not equal with respect to pd.testing.assert_frame_equal.
Specifically, they differ:

  • None values get NaN values
  • df.index changes from weird non-unique keys to a clean RangeIndex

Error message
If you rely on the same data/behavior after serialization, especially the first point causes troubles when you try to write the dataframe to a database (e.g. NaN values are not supported by SQLAlchemy)

Expected behavior
The underlying dataframes of EvaluationResult pass pd.testing.assert_frame_equal after serialization.

To Reproduce
Run test test_generative_qa_w_promptnode_eval and compare eval_result with saved_eval_result

FAQ Check

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants