This repository collects all relevant resources about interpretability in LLMs

Create feature-centric and prompt-centric visualizations for sparse autoencoders (like those from Anthropic's published research).

Latent Large Language Models

A Python framework for sequence labeling evaluation(named-entity recognition, pos tagging, etc...)

This repository contains demos I made with the Transformers library by HuggingFace.

The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.

A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.

Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.

Transformers trained on Tiny ImageNet

Python logging made (stupidly) simple

Ingest, parse, and optimize any data format ➡️ from documents to multimedia ➡️ for enhanced compatibility with GenAI frameworks

This implements training of popular model architectures, such as AlexNet, ResNet and VGG on the ImageNet dataset(Now we supported alexnet, vgg, resnet, squeezenet, densenet)

Claudette is Claude's friend

LLM101n: Let's build a Storyteller

Sparse autoencoders

Implementing a ChatGPT-like LLM in PyTorch from scratch, step by step

Let's train vision transformers (ViT) for cifar 10!

Implementation of papers in 100 lines of code.

Official implementation of "Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling"

gpt-2 from scratch in mlx

Implementation for MatMul-free LM.

Recipes for shrinking, optimizing, customizing cutting edge vision models. 💜

The Mojo Programming Language

Video+code lecture on building nanoGPT from scratch

A fast Lomb-Scargle periodogram. It's nifty, and uses a NUFFT!

Training Sparse Autoencoders on Language Models

A library for mechanistic interpretability of GPT-style language models

Deep learning at the speed of light.

use candle to implement some of the d2l.ai