Skip to content

Latest commit

 

History

History
390 lines (275 loc) · 26.8 KB

README.md

File metadata and controls

390 lines (275 loc) · 26.8 KB
Raw

Awesome-LLM-Survey

Awesome LLM Survey GitHub Repo stars

This repo aims to record survey of LLM, including instruction tuning, human alignment, LLM agent, hallucination, multi-modal, etc.

We strongly encourage the researchers that want to promote their fantastic work to the LLM survey community to make pull request to update their paper's information!

General Survey

  • A Survey of Large Language Models, 2023.11 [paper][project]

  • A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4, 2023.10 [paper]

  • Challenges and Applications of Large Language Models, 2023.07 [paper]

  • Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond, 2023.04 [paper][project]

  • A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT, 2023.02 [paper]

  • Large language models: a comprehensive survey of its applications, challenges, limitations, and future prospects, 2023.12 [paper] [project]

  • The future of gpt: A taxonomy of existing chatgpt research, current challenges, and possible future directions, 2023.04 [paper]

  • A Review on Large Language Models: Architectures, Applications, Taxonomies, Open Issues and Challenges, 2023.10 [paper]

  • Understanding LLMs: A Comprehensive Overview from Training to Inference, 2024.01 [paper]

Training of LLM

Instruction Tuning

  • Are Prompts All the Story? No. A Comprehensive and Broader View of Instruction Learning, 2023.03 [paper] [project]

  • Vision-Language Instruction Tuning: A Review and Analysis, 2023,11 [paper][project]

  • Instruction Tuning for Large Language Models: A Survey, 2023.08 [paper]

  • A Survey on Data Selection for LLM Instruction Tuning, 2024.02 [paper]

Human Alignment for LLM

  • Towards a Unified View of Preference Learning for Large Language Models: A Survey, 2024.09 [paper][project]

  • AI Alignment: A Comprehensive Survey, 2023.10 [paper][project]

  • Large Language Model Alignment: A Survey, 2023.09 [paper]

  • From Instructions to Intrinsic Human Values -- A Survey of Alignment Goals for Big Model, 2023.08 [paper][project]

  • Aligning Large Language Models with Human: A Survey, 2023.07 [paper][project]

Prompt of LLM

Chain of Thought for LLM

  • Towards Better Chain-of-Thought Prompting Strategies: A Survey, 2023.10 [paper]

  • A Survey of Chain of Thought Reasoning: Advances, Frontiers and Future, 2023.09 [paper][project]

  • Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agents, 2023.11 [paper] [project]

Prompt Engineering for LLM

  • Prompting Frameworks for Large Language Models: A Survey, 2023.11 [paper][project]

  • Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review, 2023.10 [paper]

  • Towards Better Chain-of-Thought Prompting Strategies: A Survey, 2023.10 [paper]

Retrieval-Augmented LLM

  • A Survey on Retrieval-Augmented Text Generation, 2022.02 [paper]
  • Retrieval-Augmented Generation for Large Language Models: A Survey, 2023.12 [paper] [project]
  • RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processing, 2024.04 [paper]

Challenge of LLM

Hallucination in LLM

  • Can Knowledge Graphs Reduce Hallucinations in LLMs? : A Survey, 2023.11 [paper]

  • A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions, 2023.11 [paper][project]

  • A Survey of Hallucination in “Large” Foundation Models, 2023.09 [paper][project]

  • Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models, 2023.09 [paper][project]

  • Cognitive Mirage: A Review of Hallucinations in Large Language Models, 2023.09 [paper][project]

  • Augmenting LLMs with Knowledge: A survey on hallucination prevention, 2023.09 [paper]

  • A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models, 2024.01 [paper]

  • Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignment, 2023.08 [paper]

  • Hallucination of Multimodal Large Language Models: A Survey, 2024.04 [paper]

Compression for LLM

  • A Survey on Model Compression for Large Language Models, 2023.08 [paper]
  • A Comprehensive Survey of Compression Algorithms for Language Models, 2024.01 [paper]

Evaluation of LLM

Reasoning with LLM

Explainability for LLM

  • Explainability for Large Language Models: A Survey, 2023.09 [paper]
  • The Mystery and Fascination of LLMs: A Comprehensive Survey on the Interpretation and Analysis of Emergent Abilitie, 2023.11 [paper]
  • If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agents, 2024.01 [paper]
  • From Understanding to Utilization: A Survey on Explainability for Large Language Models, 2024.01 [paper]
  • Awesome Attention Heads, 2024.07 [project]
  • Internal Consistency and Self-Feedback in Large Language Models: A Survey, 2024.07 [paper]
  • Attention Heads of Large Language Models: A Survey, 2024.09 [paper]

Fairness in LLM

  • A Survey on Fairness in Large Language Models, 2023.08 [paper]

Graph for LLM

  • A Survey of Graph Meets Large Language Model: Progress and Future Directions, 2023.11 [paper]
  • Large Language Models on Graphs: A Comprehensive Survey, 2023.12 [paper] [project]

Long-Context for LLM

  • Advancing Transformer Architecture in Long-Context Large Language Models: A Comprehensive Survey, 2023.11 [paper]
  • Length Extrapolation of Transformers: A Survey from the Perspective of Position Encoding, 2023.12 [paper]

Factuality in LLM

  • A Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity, 2023.10 [paper][project]

  • Give Me the Facts! A Survey on Factual Knowledge Probing in Pre-trained Language Models, 2023.10 [paper]

Knowledge for LLM

  • A Survey on Knowledge Distillation of Large Language Models, 2024.02 [paper]

  • Knowledge Unlearning for LLMs: Tasks, Methods, and Challenges, 2023.11 [paper]

  • Trends in Integration of Knowledge and Large Language Models: A Survey and Taxonomy of Methods, Benchmarks, and Applications, 2023.11 [paper]

  • Knowledge Editing for Large Language Models: A Survey, 2023.10 [paper]

  • Editing Large Language Models: Problems, Methods, and Opportunities, 2023.05 [paper][project]

  • Building trust in conversational ai: A comprehensive review and solution architecture for explainable, privacy-aware systems using llms and knowledge graph, 2023.08 [paper]

Self-Correction for LLM

  • Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategies, 2023.08 [paper][project]

Attributions for LLM

Tool Using of LLM

  • Foundation Models for Decision Making: Problems, Methods, and Opportunities, 2023.03 [paper]

  • Augmented Language Models: a Survey, 2023.02 [paper]

Calibration of LLM

  • A Survey of Language Model Confidence Estimation and Calibration, 2023.11 [paper]

Agent of LLM

  • A Survey on Large Language Model based Autonomous Agents, 2023.08 [paper][project]
  • The Rise and Potential of Large Language Model Based Agents: A Survey, 2023.09 [paper][project]
  • Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectives, 2023.12 [paper]
  • Large Multimodal Agents: A Survey, 2024.02 [paper][project]

Role-play of LLM

  • The Oscars of AI Theater: A Survey on Role-Playing with Language Models, 2024.09 [paper][project]
  • Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization, 2024.06 [paper][project]
  • From Persona to Personalization: A Survey on Role-Playing Language Agents, 2024.04 [paper]

Vulnerabilities of LLM

  • Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks, 2023.10 [paper]

Efficiency of LLM

  • The Efficiency Spectrum of Large Language Models: An Algorithmic Survey, 2023.12 [paper][project]

  • Efficient Large Language Models: A Survey, 2023.12 [paper][project]

  • Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment, 2023.12 [paper]

  • A Survey on Hardware Accelerators for Large Language Models, 2024.01 [paper]

  • Model Compression and Efficient Inference for Large Language Models: A Survey, 2024.02 [paper]

Data of LLM

  • Data Management For Large Language Models: A Survey, 2023.12 [paper][project]
  • A Survey on Data Selection for Language Models, 2024.02 [paper]
  • Datasets for Large Language Models: A Comprehensive Survey, 2024.02 [paper][project]

Security and Privacy of LLM

  • A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly, 2023.12 [paper]

Continual Learning of LLM

  • Continual Learning with Pre-Trained Models: A Survey, 2024.01 [paper] [project]
  • Continual Learning of Large Language Models: A Comprehensive Survey, 2024.04 [paper]

Mulitmodal of LLM

Visual LLM

  • A Survey on Benchmarks of Multimodal Large Language Models. 2024,08 [paper][project]

  • The (R)Evolution of Multimodal Large Language Models: A Survey, 2024,02 [paper]

  • Vision-Language Instruction Tuning: A Review and Analysis, 2023,11 [paper][project]

  • How to Bridge the Gap between Modalities: A Comprehensive Survey on Multimodal Large Language Model, 2023.11 [paper]

  • A Survey on Multimodal Large Language Models, 2023.06 [paper][project]

  • Multimodal Large Language Models: A Survey, 2023.11 [paper]

  • Large Language Models Meet Computer Vision: A Brief Survey, 2023.11 [paper]

  • Foundational Models Defining a New Era in Vision: A Survey and Outlook, 2023.07 [paper][project]

  • Video Understanding with Large Language Models: A Survey, 2023.12 [paper] [project]

Audio LLM

Code LLM

  • A Survey of Neural Code Intelligence: Paradigms, Advances and Beyond, 2024.03 [paper][project]
  • A Survey on Language Models for Code, 2023.11 [paper][project]
  • Pitfalls in Language Models for Code Intelligence: A Taxonomy and Survey, 2023.10 [paper][project]
  • Large Language Models Meet NL2Code: A Survey, 2022.12 [paper]
  • A Prompt Learning Framework for Source Code Summarization, 2023.12 [paper]
  • If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agents, 2024.01 [paper]

LLM for Domain Application

LLM for Health

  • A Survey of Large Language Models in Medicine: Progress, Application, and Challenge, 2023.11 [paper][project]

  • Large Language Models Illuminate a Progressive Pathway to Artificial Healthcare Assistant: A Review, 2023.10 [paper][project]

  • Large AI Models in Health Informatics: Applications, Challenges, and the Future, 2023.03 [paper][project]

  • A SWOT (Strengths, Weaknesses, Opportunities, and Threats) Analysis of ChatGPT in the Medical Literature: Concise Review, 2023.11 [paper]

  • ChatGPT in Healthcare: A Taxonomy and Systematic Review, 2023.03 [paper]

LLM for Finance

  • Large Language Models in Finance: A Survey, 2023.09 [paper]

LLM for Education

  • ChatGPT and Beyond: The Generative AI Revolution in Education, 2023.11 [paper]

LLM for Law

  • Large Language Models in Law: A Survey, 2023.12 [paper]

LLM for Mental Health

  • A review of the explainability and safety of conversational agents for mental health to identify avenues for improvement, 2023.10 [paper]
  • Towards a Psychological Generalist AI: A Survey of Current Applications of Large Language Models and Future Prospects, 2023.12 [paper]
  • Large Language Models in Mental Health Care: a Scoping Review, 2024.01 [paper]

LLM for Robotics

  • Large Language Models for Robotics: A Survey, 2023.11 [paper]

LLM for Downstream Tasks

LLM for Recommendation

  • Foundation Models for Recommender Systems: A Survey and New Perspectives, 2024.02 [paper]
  • User Modeling in the Era of Large Language Models: Current Research and Future Directions, 2023.12 [paper][project]
  • A Survey on Large Language Models for Personalized and Explainable Recommendations, 2023.11 [paper]
  • Large Language Models for Generative Recommendation: A Survey and Visionary Discussions, 2023.09 [paper]
  • A Survey on Large Language Models for Recommendation, 2023.08 [paper][project]
  • How Can Recommender Systems Benefit from Large Language Models: A Survey, 2023.06 [paper][project]

LLM for Information Retrieval

  • Large Language Models for Information Retrieval: A Survey, 2023.08 [paper][project]

LLM for Software Engineering

  • Large Language Models for Software Engineering: Survey and Open Problems, 2023.10 [paper]
  • Large Language Models for Software Engineering: A Systematic Literature Review, 2023.08 [paper]

LLM for Autonomous Driving

  • A Survey on Multimodal Large Language Models for Autonomous Driving, 2023.11 [paper]
  • A Survey of Large Language Models for Autonomous Driving, 2023.11 [paper][project]

LLM for Time Series

  • Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlook, 2023.10 [paper][project]

Detection of LLMs-Generated Content

  • A Survey on Detection of LLMs-Generated Content, 2023.10 [paper][project]
  • A Survey on LLM-generated Text Detection: Necessity, Methods, and Future Directions, 2023.10 [paper] [project]
  • Detecting ChatGPT: A Survey of the State of Detecting ChatGPT-Generated Text, 2023.09 [paper]

LLM for Society

  • Large Language Models as Subpopulation Representative Models: A Review, 2023.10 [paper]

LLM for Citation

  • When Large Language Models Meet Citation: A Survey, 2023.09 [paper]

LLM for Text Watermarking

  • A Survey of Text Watermarking in the Era of Large Language Models, 2023.12 [paper]

LLM for Math

  • Mathematical Language Models: A Survey, 2023.12 [paper]

LLM for Environmental Disciplines

  • Recent applications of AI to environmental disciplines: A review, 2023.10 [paper]
  • Opportunities and Challenges of Applying Large Language Models in Building Energy Efficiency and Decarbonization Studies: An Exploratory Overview, 2023.12 [paper]

LLM for Information Extraction

  • Large Language Models for Generative Information Extraction: A Survey, 2023.12 [paper] [project]

LLM for Data Annotation

  • Large Language Models for Data Annotation: A Survey, 2024.02 [paper] [project]

LLM for Game

  • Large Language Models and Games: A Survey and Roadmap, 2024.02 [paper]
  • A Survey on Game Playing Agents and Large Models: Methods, Applications, and Challenges, 2024.03 [paper] [project]

Star History

Star History Chart