Reading list for research topics in multimodal machine learning

[INTERSPEECH 2023] Knowledge Transfer from Pre-trained Language Models to Cif-based Recognizers via Hierarchical Distillation

Code and Pretrained Models for ICLR 2023 Paper "Contrastive Audio-Visual Masked Autoencoder".

This repository is an unoffical PyTorch implementation of Semi-Supervised Learning on Medical Segmentation in 2D and 3D.

A library for speech data augmentation in time-domain

✨✨Latest Advances on Multimodal Large Language Models

Implementation of 🦩 Flamingo, state-of-the-art few-shot visual question answering attention net out of Deepmind, in Pytorch

Promting Whisper for Audio-Visual Speech Recognition, Code-Switched Speech Recognition, and Zero-Shot Speech Translation

INTERSPEECH 2023-2024 Papers: A complete collection of influential and exciting research papers from the INTERSPEECH 2023-24 conference. Explore the latest advances in speech and language processin…

FAIR Sequence Modeling Toolkit 2

Convert Machine Learning Code Between Frameworks

Learning audio concepts from natural language supervision

Rectified Rotary Position Embeddings

Code and Pretrained Models for Interspeech 2023 Paper "Whisper-AT: Noise-Robust Automatic Speech Recognizers are Also Strong Audio Event Taggers"

This repository is an unoffical PyTorch implementation of Medical segmentation in 2D and 3D.

Arxiv个性化定制化模版，实现对特定领域的相关内容、作者与学术会议的有效跟进。

Open source core of Synergy, the cross-platform keyboard and mouse sharing tool (Windows, macOS, Linux)

Production First and Production Ready End-to-End Speech Recognition Toolkit

Foundation Architecture for (M)LLMs

An implementation of "Retentive Network: A Successor to Transformer for Large Language Models"

Official release of InternLM2.5 base and chat models. 1M context support

Official repository of OFA (ICML 2022). Paper: OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework

IFSeg: Image-free Semantic Segmentation via Vision-Language Model (CVPR 2023)

Source code for paper "Improving Task-Specific Generalization in Few-Shot Learning via Adaptive Vicinal Risk Minimization"

Central place for the engineering/scaling WG: documentation, SLURM scripts and logs, compute environment and data.

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Fine-tune the Whisper speech recognition model to support training without timestamp data, training with timestamp data, and training without speech data. Accelerate inference and support Web deplo…

A playbook for systematically maximizing the performance of deep learning models.

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities