Open source platform for the machine learning lifecycle

Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"

A algebraic word problem dataset, with multiple choice questions annotated with rationales.

Instruct-tune LLaMA on consumer hardware

Code for our EMNLP 2023 Paper: "LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models"

A curated list for Efficient Large Language Models

🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

[TMLR 2024] Efficient Large Language Models: A Survey

👩‍💻👨‍💻 AI 엔지니어 기술 면접 스터디 (⭐️ 1k+)

Inference code for LLaMA models in JAX

Large language models (LLMs) made easy, EasyLM is a one stop solution for pre-training, finetuning, evaluating and serving LLMs in JAX/Flax.

tiktoken is a fast BPE tokeniser for use with OpenAI's models.

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries

Pipeline Parallelism for PyTorch

PyTorch extensions for high performance and large scale training.

Implementation of Rotary Embeddings, from the Roformer paper, in Pytorch

A modern model graph visualizer and debugger

Distributed Evolutionary Algorithms in Python

Code Transformer neural network components piece by piece

Google Research

[ACL'20] HAT: Hardware-Aware Transformers for Efficient Natural Language Processing

Curation note of NLP datasets

Minimal, clean code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization.

Abseil Common Libraries (Python)

Datasets, Transforms and Models specific to Computer Vision

Kolmogorov Arnold Networks

Ongoing research training transformer models at scale

Flexible and powerful tensor operations for readable and reliable code (for pytorch, jax, TF and others)

JAX Synergistic Memory Inspector

Everything you want to know about Google Cloud TPU