DataComp for Language Models

DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence

MoRA: High-Rank Updating for Parameter-Efficient Fine-Tuning

Mixture-of-Experts (MoE) Language Model

llama3 implementation one matrix multiplication at a time

My favorite C programming practices.

DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model

DeepSeek LLM: Let there be answers

Kolmogorov Arnold Networks

Official repository for ORPO

RewardBench: the first evaluation tool for reward models.

Robust recipes to align language models with human and AI preferences

Official repository for paper "Weak-to-Strong Extrapolation Expedites Alignment"

OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.

Sailor: Open Language Models for South-East Asia

a huggingface mirror site.

The official Meta Llama 3 GitHub site

ReFT: Representation Finetuning for Language Models

Official repository of Evolutionary Optimization of Model Merging Recipes

Open-Sora: Democratizing Efficient Video Production for All

Grok open release

Source code of "Task arithmetic in the tangent space: Improved editing of pre-trained models".

20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference. Documentation:

Implementing a ChatGPT-like LLM in PyTorch from scratch, step by step

Reference implementation for DPO (Direct Preference Optimization)