Self-Supervised Speech Pre-training and Representation Learning Toolkit

PyTorch implementation of "data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language" from Meta AI

This repository contains code for applying Data2Vec to pretrain Keyword Transformer model as described in "Improving Label-Deficient Keyword Spotting Through Self-Supervised Pretraining".

PyTorch implementation of Data2Vec self-supervised approach for vision use cases.

Ready to run PyTorch implementation of Data2Vec 2.0: Highly efficient self-supervised representation learning for vision, speech and text.

Deep Learning course final project at ENSAE Paris. The project is centered around data2vec model (first multi-modal self-supervised algorithm). The goal is to compare it to unimodal self-supervised models on Question Answering and Sentiment Classification tasks.