Meta Lingua: a lean, efficient, and easy-to-hack codebase to research LLMs.

TPU pod commander is a package for managing and launching jobs on Google Cloud TPU pods.

Pax is a Jax-based machine learning framework for training large scale models. Pax allows for advanced and fully configurable experimentation and parallelization, and has demonstrated industry lead…

Fixes mojibake and other glitches in Unicode text, after the fact.

Scalable data pre processing and curation toolkit for LLMs

A Jax-based library for designing and training transformer models from scratch.

Legible, Scalable, Reproducible Foundation Models with Named Tensors and Jax

A simple, performant and scalable Jax LLM!

Accelerate, Optimize performance with streamlined training and serving options with JAX.

Large language models (LLMs) made easy, EasyLM is a one stop solution for pre-training, finetuning, evaluating and serving LLMs in JAX/Flax.

Implementation of the training framework proposed in Self-Rewarding Language Model, from MetaAI

Convert PDF to markdown quickly with high accuracy

The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.

Utility for behavioral and representational analyses of Language Models

LLM inference in C/C++

Repo for the Belebele dataset, a massively multilingual reading comprehension dataset.

ByT5 model scripts

The RedPajama-Data repository contains code for preparing large datasets for training large language models.

The collection of files that contain datasets for 6 languages (Russian, Basque, Turkish, Spanish, Czech and English) with labels of different morphological complexity

List of papers on hallucination detection in LLMs.

Code and data for the paper "Evaluating German Transformer Language Models with Syntactic Agreement Tests" (Zaczynska et al., 2020)

Large language models (LLMs) made easy, EasyLM is a one stop solution for pre-training, finetuning, evaluating and serving LLMs in JAX/Flax.

Code for Paper: “Low-Resource” Text Classification: A Parameter-Free Classification Method with Compressors

The simplest, fastest repository for training/finetuning medium-sized GPTs.

Due to restriction of LLaMA, we try to reimplement BLOOM-LoRA (much less restricted BLOOM license here https://huggingface.co/spaces/bigscience/license) using Alpaca-LoRA and Alpaca_data_cleaned.json

Expanding natural instructions

LTG-Bert

Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".

Reduce the size of pretrained Hugging Face models via vocabulary trimming.

Summarization Papers