The official repo for "LLoCo: Learning Long Contexts Offline"

[EMNLP 2023] Adapting Language Models to Compress Long Contexts

Multilingual Sentence & Image Embeddings with BERT

Fast Differentiable Tensor Library in JavaScript and TypeScript with Bun + Flashlight

TensorDict is a pytorch dedicated tensor container.

Code for "Merging Text Transformers from Different Initializations"

⛷️ LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training

Implementation of ST-Moe, the latest incarnation of MoE after years of research at Brain, in Pytorch

Tile primitives for speedy kernels

Run your GitHub Actions locally 🚀

A library for mechanistic interpretability of GPT-style language models

Doing simple retrieval from LLM models at various context lengths to measure accuracy

Instant voice cloning by MyShell.

🧑‍🏫 60 Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gan…

A framework for conducting machine learning experiments in python

SONAR, a new multilingual and multimodal fixed-size sentence embedding space, with a full suite of speech and text encoders and decoders.

Localized watermarking for AI-generated speech audios, with SOTA on robustness and very fast detector

Facebook AI Research Sequence-to-Sequence Toolkit written in Python.

FAIR Sequence Modeling Toolkit 2

Foundational Models for State-of-the-Art Speech and Text Translation

3-days AI hackathon by 1337AI and Math&Maroc

Code and documentation to train Stanford's Alpaca models, and generate the data.

MuAViC: A Multilingual Audio-Visual Corpus for Robust Speech Recognition and Robust Speech-to-Text Translation

LaTeX Template for Mike Morrison's #betterposter

Robust Speech Recognition via Large-Scale Weak Supervision

Code and Data release for "Improving Multilingual Translation by Representation and Gradient Regularization" (Yang et al. EMNLP 2021)

📖 A curated list of resources dedicated to Natural Language Processing (NLP)

YSDA course in Natural Language Processing

Solve puzzles. Improve your pytorch.