BABILong is a benchmark for LLM evaluation using the needle-in-a-haystack approach.

Doing simple retrieval from LLM models at various context lengths to measure accuracy

OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.

This repo includes ChatGPT prompt curation to use ChatGPT better.

[ICML 2024] TrustLLM: Trustworthiness in Large Language Models

The official GitHub page for the survey paper "A Survey on Evaluation of Large Language Models".

An Open Robustness Benchmark for Jailbreaking Language Models [arXiv 2024]

Robust recipes to align language models with human and AI preferences

A quick guide (especially) for trending instruction finetuning datasets

A library for mechanistic interpretability of GPT-style language models

A very simple framework for state-of-the-art Natural Language Processing (NLP)

Papers and resources related to the security and privacy of LLMs 🤖

Model interpretability and understanding for PyTorch

Interpretability for sequence generation models 🐛 🔍

The jailbreak-evaluation is an easy-to-use Python package for language model jailbreak evaluation.

A reading list for large models safety, security, and privacy (including Awesome LLM Security, Safety, etc.).

[NAACL2024] Attacks, Defenses and Evaluations for LLM Conversation Safety: A Survey

A curation of awesome tools, documents and projects about LLM Security.

Open-Sora: Democratizing Efficient Video Production for All

Code and data for "The Power of Noise: Redefining Retrieval for RAG Systems"

🔍 LLM orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your d…

Efficient Retrieval Augmentation and Generation Framework

A framework for few-shot evaluation of language models.

Python Fire is a library for automatically generating command line interfaces (CLIs) from absolutely any Python object.

Typed argument parser for Python

ICLR2024 Paper. Showing properties of safety tuning and exaggerated safety.

Capture Screen, Audio, Cursor, Mouse Clicks and Keystrokes

The papers are organized according to our survey: Evaluating Large Language Models: A Comprehensive Survey.