ITM threshold

[chaochen99]

HI, I am very fortunate to read your paper BLIP. It's very exciting.

I wonder how to set the ITM threshold when filtering?

Thanks!
Looking forward to your reply.

[LiJunnan1992]

We simply set it as 0.5, thanks.

[chaochen99]

In train_retrieval.py, the score of ITM did not go through softmax. Did you consider using softmax at that time?
When filtering, is the score of 0.5 used without softmax?

Thanks!

[LiJunnan1992]

When filtering, the score of 0.5 uses softmax?

During inference of retrieval, softmax does not affect the ranking of ITM predictions, but it does affect the relative scale w.r.t the ITC similarity. If softmax is used for ITM scores, some additional scaling on the ITC score is also needed.