request for the data in Image Captioning experiment

[ChrisRBXiong]

Hi, would you mind release the data used in Image Captioning experiment (human judgments of twelve submission entries from the COCO 2015 Captioning Challenge) in your paper? Thanks a lot!

[felixgwu]

Hi @ChrisRBXiong,

We follow the setup of Learning to Evaluate Image Captioning.
The human scores are from the COCO leaderboard. You'll see the M1 & M2 human scores if you click the "Challenge2015" button.
You may contact the authors if you have additional questions regarding their setting.

Best,
Felix

[GaryYufei]

Hi, just a follow here. Any idea about how to get those system outputs?

[felixgwu]

Hi @GaryYufei,

Please contact the authors of Learning to Evaluate Image Captioning for the data set and experimental setup.

Best,
Felix

[GaryYufei]

Thanks Felix. Yufei

…

On 5 Jun 2020, at 3:31 pm, felixgwu ***@***.***> wrote: Hi @GaryYufei <https://github.com/GaryYufei>, Please contact the authors of Learning to Evaluate Image Captioning <https://arxiv.org/abs/1806.06422> for the data set and experimental setup. Best, Felix — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#22 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB7HVRKYZOSISPDGABUIWTLRVB7JNANCNFSM4JP7O6FQ>.

[GaryYufei]

I am sorry @felixgwu, but I got no reply from the authors of "Learning to Evaluate Image Captioning". I think I understand all the setup from that paper and the only missing data is the system outputs on COCO 2014 Val Set (I think it should here, but it seems to be down). Any pointers?

[felixgwu]

Hi @GaryYufei, Here is a link to the val set outputs: https://drive.google.com/file/d/1HUrcgLXTNUY9ZbJ6eegb2V92Hfh17j9e
I hope this helps.

[GaryYufei]

Cool. That's exactly what I want! Thank you very much

[PierreColombo]

@GaryYufei were you able to reproduce the results ?
I got 40k captions where in the MoverScore paper they mention 5k examples

[yavuzdrmzksr]

Hi @felixgwu, I downloaded the val set outputs from the link you provided (https://drive.google.com/file/d/1HUrcgLXTNUY9ZbJ6eegb2V92Hfh17j9e). As far as I can see, "junhua.mao" and "mRNN_share.JMao" submissions have the same outputs. Are those submissions correct? In Figure 7 of "Learning to Evaluate Image Captioning" paper they state they compute different results for "m-RNN" and "m-RNN (Baidu/ UCLA)".