Code for The Annotated Transformer blog post:

https://nlp.seas.harvard.edu/2018/04/03/attention.html

We, SkoltechNLP group, revisited this notebook in April 2021 and made some adjusments to make it runnable like "Run All Cells" with updated required packages.

Aslo, we add some clarifing images from the amazing resources about Transformers that emerged during these years:

• Seq2Seq and Attention by Lena Voita
• The Illustrated Transformer

We believe that will add some additional understanding of the main idea and code. Enjoy your Transformer!

• Daryna Dementieva (@dementyeva_ds in any social network or [email protected])
• Anton Razzhigaev ([email protected])
• Alexander Panchenko ([email protected])