A fineturned BERT on GoEmotions with Huggingface Transformers.

1. Based on the uncased BERT pretrained model with a linear output layer.
2. Added several commonly-used emoji and tokens to the special token list of the tokenizer.
3. Used weighted loss and focal loss with label smoothing while training.

Make a directory for saving models.

$ mkdir model
$ cd model

Download models from Huggingface.

$ git clone https://huggingface.co/justin871030/bert-base-uncased-goemotions-original-finetuned
$ git clone https://huggingface.co/justin871030/bert-base-uncased-goemotions-ekman-finetuned
$ git clone https://huggingface.co/justin871030/bert-base-uncased-goemotions-group-finetuned

Pretrained Model: BERT-Base, Uncased (12-layer, 768-hidden, 12-heads)

Best Results of Macro F1

• GoEmotions
• GoEmotions Github
• Huggingface Transformers
• GoEmotions-PyTorch