A comprehensive framework to explore whether embodied multimodal models are plausibly resilient

Egocentric Video Understanding Dataset (EVUD)

Repository of PIXAR, a Pixel-based Auto-Regressive Language Model

A Framework for the Systematic Evaluation of Chat-Optimized Language Models as Conversational Agents and an Extensible Benchmark

A framework for few-shot evaluation of vision+language models.

Create efficient datasets for training Multimodal Foundation Models

Model code for Embodied MultiModal Agent (EMMA)

A Data Streaming Library for Efficient Neural Network Training

A web app for ranking computer science departments according to their research output in selective venues, and for finding active faculty across a wide range of areas.

Code for EmBERT, a transformer model for embodied, language-guided visual task completion.

TEACh is a dataset of human-human interactive dialogues to complete tasks in a simulated household environment.

🤗 Transformers: State-of-the-art Natural Language Processing for Pytorch, TensorFlow, and JAX.

The lightweight PyTorch wrapper for high-performance AI research. Scale your models, not the boilerplate.

Starter Code for VALUE benchmark

ALFRED - A Benchmark for Interpreting Grounded Instructions for Everyday Tasks

A python wrapper for the Visual Genome API

PyTorch bottom-up attention with Detectron2

A very simple framework for state-of-the-art Natural Language Processing (NLP)

A fast and simple framework for building and running distributed applications. Ray is packaged with RLlib, a scalable reinforcement learning library, and Tune, a scalable hyperparameter tuning libr…

A toolkit for developing and comparing reinforcement learning algorithms.

GuessWhat?! Baselines

Generic package with basic tools and abstract class to run experiments

A modular framework for vision & language multimodal research from Facebook AI Research (FAIR)

🤗 nlp: datasets and evaluation metrics for Natural Language Processing in NumPy, Pandas, PyTorch and TensorFlow

Feature extraction and visualization scripts for nocaps baselines.

Code for the paper "VisualBERT: A Simple and Performant Baseline for Vision and Language"

Homebrew formulas

Vision-Language Pre-training for Image Captioning and Question Answering