Repo for external large-scale work

Code associated with the paper "Entropy-based Attention Regularization Frees Unintended Bias Mitigation from Lists"

Implementation of RETRO, Deepmind's Retrieval based Attention net, in Pytorch

Open-source pre-training implementation of Google's LaMDA in PyTorch. Adding RLHF similar to ChatGPT.

Multi-layer Recurrent Neural Networks (LSTM, GRU, RNN) for character-level language models in Torch

Easy-Translate is a script for translating large text files with a SINGLE COMMAND. Easy-Translate is designed to be as easy as possible for beginners and as seamlesscustomizable and as possible for…

BERT score for text generation

UmBERTo: an Italian Language Model trained with Whole Word Masking.

Stanford NLP Python library for tokenization, sentence segmentation, NER, and parsing of many human languages

GLM-130B: An Open Bilingual Pre-Trained Model (ICLR 2023)

Helsinki Prosody Corpus and A System for Predicting Prosodic Prominence from Text

Code for the paper "Language Models are Unsupervised Multitask Learners"

Language Modeling with the H3 State Space Model

RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference,…

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries

DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature

Making large AI models cheaper, faster and more accessible

An unnecessarily tiny implementation of GPT-2 in NumPy.

Official implementation for "Multimodal Chain-of-Thought Reasoning in Language Models" (stay tuned and more will be updated)

Running large language models on a single GPU for throughput-oriented scenarios.

tiktoken is a fast BPE tokeniser for use with OpenAI's models.

Utilities to use the Hugging Face Hub API

Unified embedding generation and search engine. Also available on cloud - cloud.marqo.ai

LLM inference in C/C++

The simplest way to run LLaMA on your local machine

Code and documentation to train Stanford's Alpaca models, and generate the data.

C++ implementation for BLOOM

State-of-the-art Machine Learning for the web. Run 🤗 Transformers directly in your browser, with no need for a server!

Instruct-tune LLaMA on consumer hardware