Why is there no model checkpoint that perform ITC+ITM+LM Loss on Coco/Flickr?

[linzhiqiu]

I am curious why BLIP does not choose to apply all 3 losses when finetuning on COCO/Flickr. I would have thought that using all 3 losses will produce a model that can simultaneously perform both retrieval and captioning (on COCO/Flickr). Let me know if this is an mis-understanding!

[Pppapaya]

I am curious why BLIP does not choose to apply all 3 losses when finetuning on COCO/Flickr. I would have thought that using all 3 losses will produce a model that can simultaneously perform both retrieval and captioning (on COCO/Flickr). Let me know if this is an mis-understanding!

Hi, I'm curious about it too, do you know the reason now?

[linzhiqiu]

I think BLIP-2 answers this question already -- BLIP-2 trained with 3 losses and got improved performance

[gwyong]

What does it mean? I couldn't understand. Does it mean BLIP was trained with three losses separately or trained together? I think it only used LM loss for fine tuning in image captioning,

[BrianG13]

@linzhiqiu any progress on this?