IMDB sentiment classification with GPT2 pre training. The main file is here

You can refresh your knowledge about GPT2 from this awesome blog

you can run a similar notebook in colab from here

what is a cls token? it is a token for classification. BERT has this token and it is retrieved from pooling and some other functions such as sigmoid, etc. it is not precisely said in the paper (if you found it please show me so i can edit this). some other folks on github found that it is ok to average each word vector to make and CLS token. so we are adapting to this approach here.

• Base notebook

• imdb without training

• gpt 2 docs from hugging face (and it's pre train)

• kaggle data sets from

• jay alammar blog for visualizing gpt2

• a stack overflow to remove html tags from the data set