Question about the CapFilt method

[xiaovhua]

Thank you for your impressive work!

If possible, I want to ask some details about the CapFilt.

I want to apply CapFilt on my noise dataset for pre-process and augmentation. However, I notice that no separate captioner and filter models are access in the repository. As an alternative, can I use the checkpoint of Image Caption FT from COCO as captioner and checkpoint of Retrieval FT from COCO as filter (use the cos similarity as score), or should I use the BILP with CapFilt directly? Will the former works better since it trains from the filtered dataset?

Thanks!

[LiJunnan1992]

Yes you can use the caption checkpoint and the retrieval checkpoint from COCO as the captioner and filter. Thanks!