Reading NLP-related papers when working, summary and thoughts. Major in Machine Reading Comprehension (MRC), Question Answering (QA), Multi-document MRC, Information Retriever (IR) and Ranking.

• LSA, references in Wikipedia, An Introduction to Latent Semantic Analysis, Indexing by Latent Semantic Analysis, CUHK Latent Semantic Indexing (LSI) An Example, CUHK PPT, Stanford NLP IR-book latent-semantic-indexing.
• LDA, references in Wikipedia, LDA Topic Modeling: An Explanation, Topic Modeling and Latent Dirichlet Allocation (LDA) in Python, Paper Latent Dirichlet Allocation, Your Guide to Latent Dirichlet Allocation, Topic modeling using Latent Dirichlet Allocation(LDA) and Gibbs Sampling explained!, Topic Modeling with Gensim (Python).
• SVD, references in Wikipedia,

Visiting Tomas Mikolov Google Scholar to get related papers.

Other papers list:

• A Neural Probabilistic Language Model

BERT-style Models, Summary PPT.

• BERT
• XLNet
• structBERT
• spanBERT
• SemBERT
• RoBERTa
• ALBERT

Summary PPT.

• RE3QA
• RankQA

KD is similar to Label Smoothing, introduction of KD from the following blogs:

• Neural Network Distiller
• Smaller, faster, cheaper, lighter: Introducing DistilBERT, a distilled version of BERT
• Tap into the dark knowledge using neural nets — Knowledge distillation
• Knowledge Distillation

MS MARCO LeaderBoard, several public paper models, written on Github MSMARCO-MRC-Analysis

• Duet: [arxiv], [PPT], [notes]
• Duet V2

• Adam TODO

• 《Deep Learning》
• 《Deep Learning (中文版)》
• 《Mathematics for Machine Learning》
• 《Learning to rank for information retrieval》
• 《GDL-Generative Deep Learning》
• 《Introduction to Information Retrieval》
• 《word2vec math theories》
• 《TensorFlow2深度学习》 [code]