Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allow switching between confidence scores in reader #1515

Closed
julian-risch opened this issue Jun 23, 2021 · 0 comments · Fixed by deepset-ai/FARM#836
Closed

allow switching between confidence scores in reader #1515

julian-risch opened this issue Jun 23, 2021 · 0 comments · Fixed by deepset-ai/FARM#836
Assignees

Comments

@julian-risch
Copy link
Member

Let's have only one score in answers but support switching between the two options: probability and score
To this end, the prediction head needs to sort based on a switch here:
https://github.com/deepset-ai/FARM/blob/cf6f96ffcedeb7b1341278dcdd6437e99efaa75b/farm/modeling/prediction_head.py#L1467
Right now, answers contain both probability and score:

{'answer': 'Eddard',
 'context': 's Nymeria after a legendary warrior queen. '
            'She travels with her father, Eddard, to '
            "King's Landing when he is made Hand of the "
            'King. Before she leaves,',
 'document_id': 'ba2a8e87ddd95e380bec55983ee7d55f',
 'meta': {'name': '43_Arya_Stark.txt'},
 'offset_end': 78,
 'offset_end_in_doc': 153,
 'offset_start': 72,
 'offset_start_in_doc': 147,
 'probability': 0.9899835586547852,
 'score': 15.927592277526855}

Related issue in Haystack #1220

@Timoeller Timoeller transferred this issue from deepset-ai/FARM Sep 28, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant