Odyssey: Empowering Agents with Open-World Skills

Agentic components of the Llama Stack APIs

A family of compressed models obtained via pruning and knowledge distillation

Container plugin for Slurm Workload Manager

A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)

MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.

Grok open release

Tmux configuration, that supercharges your tmux to build cozy and cool terminal environment

Rectified Rotary Position Embeddings

Llama3、Llama3.1 中文仓库（聚合资料，各种网友及厂商微调、魔改版本有趣权重 & 训练、推理、评测、部署教程视频 & 文档）

🚀 Accelerate training and inference of 🤗 Transformers and 🤗 Diffusers with easy to use hardware optimization tools

TensorRT Model Optimizer is a unified library of state-of-the-art model optimization techniques such as quantization and sparsity. It compresses deep learning models for downstream deployment frame…

QQQ is an innovative and hardware-optimized W4A8 quantization solution.

Code for paper: "QuIP: 2-Bit Quantization of Large Language Models With Guarantees"

YaRN: Efficient Context Window Extension of Large Language Models

[COLM 2024] TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding

A pytorch quantization backend for optimum

A quantization algorithm for LLM

KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache

对llama3进行全参微调、lora微调以及qlora微调。

A library for easily merging multiple LLM experts, and efficiently train the merged LLM.

A repository dedicated to evaluating the performance of quantizied LLaMA3 using various quantization methods..

OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.

The papers are organized according to our survey: Evaluating Large Language Models: A Comprehensive Survey.

FacTool: Factuality Detection in Generative AI

Must-read Papers on LLM Agents.

Deita: Data-Efficient Instruction Tuning for Alignment [ICLR2024]

[ACL'24 Oral] Data and code for L-Eval, a comprehensive long context language models evaluation benchmark