RoBERTa on SuperGLUE's 'Winogender Schema Diagnostics' task

[dirkgr]

Winogender Schema Diagnostics (AX-g) is one of the tasks of the SuperGLUE benchmark. The task is to re-trace the steps of Facebook's RoBERTa paper (https://arxiv.org/pdf/1907.11692.pdf) and build an AllenNLP config that reads the AX-g data and fine-tunes a model on it. We expect scores in the range of their entry on the SuperGLUE leaderboard.

This can be formulated as a classification task, using the TransformerClassificationTT model, analogous to the IMDB model. You can start with the experiment config and dataset reading step from IMDB, and adapt them to your needs.