Meta Lingua: a lean, efficient, and easy-to-hack codebase to research LLMs.

Tips for Writing a Research Paper using LaTeX

DSPy: The framework for programming—not prompting—language models

Code for paper "The effect of batch size on contrastive self-supervised speech representation learning"

Minimalistic large language model 3D-parallelism training

20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

Fast lexical search implementing BM25 in Python using Numpy, Numba and Scipy

A repository for research on medium sized language models.

Structured Text Generation

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

JAX-Toolbox

[TPAMI'24] Hi-SAM: Marrying Segment Anything Model for Hierarchical Text Segmentation

A native PyTorch Library for large model training

A repository for managing public, versioned releases of the Swedish Parliament Corpus.

The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models"

My Arch dotfiles

Zero-Shot Speech Editing and Text-to-Speech in the Wild

Web Serving and Remote Procedure Calls at 50x lower latency and 70x higher bandwidth than FastAPI, implementing JSON-RPC & REST over io_uring ☎️

Fast Open-Source Search & Clustering engine × for Vectors & 🔜 Strings × in C++, C, Python, JavaScript, Rust, Java, Objective-C, Swift, C#, GoLang, and Wolfram 🔍

PyTorch native finetuning library

docTR (Document Text Recognition) - a seamless, high-performing & accessible library for OCR-related tasks powered by Deep Learning.

🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash attention v2.

library supporting NLP and CV research on scientific papers

A pipeline to improve skills of large language models

Experiments for efforts to train a new and improved t5