MAP and MRR wrong for multiple gold documents

[ju-gu]

Describe the bug
Both the MAP and the MRR show wrong values. It seems we calculate the score for single gold documents and then override it after each instead of calculating it for the whole batch of gold documents.

Expected behavior
correct values

To Reproduce

retrieved_docs = ["one", "two", "three", "four", "five", "six", "seven", "eight", "nine", "ten"]
gold_docs = ["one", "two", "three","four","seven"]

from haystack.components.evaluators import DocumentMAPEvaluator, DocumentMRREvaluator
from haystack import Document

mapevaluator = DocumentMAPEvaluator()
mrrevaluator = DocumentMRREvaluator()
mapresult = mapevaluator.run(
    ground_truth_documents=[[Document(content=content) for content in gold_docs]],
    retrieved_documents=[[Document(content=content) for content in retrieved_docs]])

mrrresult = mrrevaluator.run(
    ground_truth_documents=[[Document(content=content) for content in gold_docs]],
    retrieved_documents=[[Document(content=content) for content in retrieved_docs]])

print(mapresult["individual_scores"])
print(mrrresult["individual_scores"])
print(mapresult["score"])
print(mrrresult["score"])

[mrm1001]

Hi @ju-gu I'm trying to understand the example above. You are providing 10 "retrievals" but have only 5 gold truth docs? In other words, you have:

Actual retrieved	True doc
one	one
two	two
three	three
four	four
five	seven
six	?
seven	?
eight	?
nine	?
ten	?

I don't think this is a typical use case, are you sure this is what you're trying to do?

Or are you trying to do this instead:

Actual retrieved	True doc
["one", "two", "three", "four", "five", "six", "seven", "eight", "nine", "ten"]	["one", "two", "three","four","seven"]

[ju-gu]

I am doing the latter. The two lists retrieved docs and true docs become a list of lists in these lines:

ground_truth_documents=[[Document(content=content) for content in gold_docs]]
retrieved_documents=[[Document(content=content) for content in retrieved_docs]])

So it is just one set of retrieved docs and one set of gold documents