Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EvalationResult.calculate_metrics() returns numpy.float32 instead of float as metric value for SAS #3590

Closed
1 task done
tstadel opened this issue Nov 16, 2022 · 3 comments · Fixed by #3612
Closed
1 task done
Assignees
Labels
topic:eval type:bug Something isn't working

Comments

@tstadel
Copy link
Member

tstadel commented Nov 16, 2022

Describe the bug
EvalutionResult.calculate_metrics() returns sas metric as numpy.float32 type and not as float.

Expected behavior
Return type is float as the type hint of EvalutionResult.calculate_metrics().

Additional context
numpy.float32 cannot be used for storage in databases, e.g. with sqlalchemy.

To Reproduce

  1. run tutorial 5 with sas metrics
  2. look for sas metrics value type.

FAQ Check

System:

  • Haystack version (commit or version number): 1.10
@tstadel tstadel added type:bug Something isn't working topic:eval labels Nov 16, 2022
@julian-risch
Copy link
Member

Hi @tstadel would you like to contribute a PR for this issue yourself or should our core engineering team take over? 🙂 At first glance, it looks like a small issue. Either way, just let me know. Thank you!

@tstadel
Copy link
Member Author

tstadel commented Nov 21, 2022

@julian-risch I have to check again, because it seems like all metric values are returned as numpy types.

@tstadel
Copy link
Member Author

tstadel commented Nov 21, 2022

Yes, and for some SAS models it might be numpy.float32 and not numpy.64. So let's consolidate all outputs of calculate_metrics() to standard python float.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic:eval type:bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants