A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)

A playbook for systematically maximizing the performance of deep learning models.

Performance instrumentation and tracing for Android, Linux and Chrome (read-only mirror of https://android.googlesource.com/platform/external/perfetto/)

A CPU+GPU Profiling library that provides access to timeline traces and hardware performance counters.

NCCL Tests

Ongoing research training transformer models at scale

Hackable and optimized Transformers building blocks, supporting a composable construction.

Fairring (FAIR + Herring) is a plug-in for PyTorch that provides a process group for distributed training that outperforms NCCL at large scales

Mass-editing thousands of facts into a transformer memory (ICLR 2023)

Framework code with wandb, checkpointing, logging, configs, experimental protocols. Useful for fine-tuning models or training from scratch, and testing them on a variety of datasets (transfer learn…

Legible, Scalable, Reproducible Foundation Models with Named Tensors and Jax

[NeurIPS 2023, ICMI 2023] Quantifying & Modeling Multimodal Interactions

This is a list of peer-reviewed representative papers on deep learning dynamics (optimization dynamics of neural networks). The success of deep learning attributes to both network architecture and …

The official repository for the paper "Revisiting pre-trained remote sensing model benchmarks: resizing and normalization matters"

A collection aiming to bring all the amazing work of Deep Learning for Climate Super-res together.

Recipes for reproducing Analysis-Ready & Cloud Optimized (ARCO) ERA5 datasets.

Hybrid ML + physics model of the Earth's atmosphere

SKAI is a machine learning based tool for performing automatic building damage assessments on aerial imagery of disaster sites.

A benchmark for the next generation of data-driven global weather models.

RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference,…

Corruption and Perturbation Robustness (ICLR 2019)

disentanglement_lib is an open-source library for research on learning disentangled representations.

Dataset to assess the disentanglement properties of unsupervised learning methods

An Extendible (General) Continual Learning Framework based on Pytorch - official codebase of Dark Experience for General Continual Learning