Efficient Triton Kernels for LLM Training

Codebase for Merging Language Models (ICML 2024)

System prompts from Apple's new Apple Intelligence on MacOS Sequoia

[ICCV 2017] Torch code for Grad-CAM

LoRA-Composer: Leveraging Low-Rank Adaptation for Multi-Concept Customization in Training-Free Diffusion Models

Building modular LMs with parameter-efficient fine-tuning.

Pytorch implementation of the PEER block from the paper, Mixture of A Million Experts, by Xu Owen He at Deepmind

A collection of AWESOME things about mixture-of-experts

Implementation of the paper: "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"

The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery 🧑‍🔬

Code for paper: "MeteoRA: Multiple-tasks Embedded LoRA for Large Language Models"

Awesome-LLM: a curated list of Large Language Model

LaTeX style file for the Journal of Machine Learning Research

Code and example data for the paper: Rule Based Rewards for Language Model Safety

A simple pip-installable Python tool to generate your own HTML citation world map from your Google Scholar ID.

Repository for Meta Chameleon, a mixed-modal early-fusion foundation model from FAIR.

Bridging LLM and Recommender System.

Parameter-Efficient Sparsity Crafting From Dense to Mixture-of-Experts for Instruction Tuning on General Tasks

State-of-the-art Parameter-Efficient MoE Fine-tuning Method

Codebase for reproducing the experiments of the semantic uncertainty paper (short-phrase and sentence-length experiments).

Fast Matrix Multiplications for Lookup Table-Quantized LLMs

Official PyTorch Code for SAFF (ICCV 2023)

Sparse Autoencoder for Mechanistic Interpretability

Using sparse coding to find distributed representations used by neural networks.

The official implementation of the paper "Demystifying the Compression of Mixture-of-Experts Through a Unified Framework".