The attention heads in the Transformer architecture possess a variety of capabilities. This is a carefully compiled list that summarizes the diverse functions of the attention heads.

Resources for skilling up in AI alignment research engineering. Covers basics of deep learning, mechanistic interpretability, and RL.

Code for the paper "Knowledge Rumination for Pre-trained Language Models."

Must-read Papers on Model Editing.

Code for the paper "Schema-aware Reference as Prompt Improves Data-Efficient Relational Triple and Event Extraction"

CogKTR: A Knowledge-Enhanced Text Representation Toolkit for Natural Language Understanding

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

Paper collections of retrieval-based(augmented) language model.

Code for the paper "Kformer: Knowledge Injection in Transformer Feed-Forward Layers"

2015年以来基于深度学习方法的事件抽取论文整理

Pyserini is a Python toolkit for reproducible information retrieval research with sparse and dense representations.

A summary of must-read papers for Neural Question Generation (NQG)

💭 Aspect-Based-Sentiment-Analysis: Transformer & Explainable ML (TensorFlow)

Awesome Knowledge-Distillation. 分类整理的知识蒸馏paper(2014-2021)。

A collection of AWESOME things about domian adaptation

🔍 Transformers at scale for question answering & neural search. Using NLP via a modular Retriever-Reader-Pipeline. Supporting DPR, Elasticsearch, HuggingFace's Modelhub...