-
Stability.ai, Eleuther.ai
- Seattle, WA
- https://dmarx.github.io
- @DigThatData
Multi-modal
Align and Prompt: Video-and-Language Pre-training with Entity Prompts
Official implementations for various pre-training models of ERNIE-family, covering topics of Language Understanding & Generation, Multimodal Understanding & Generation, and beyond.
PyTorch code for BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
[CVPR 2022] Show Me What and Tell Me How: Video Synthesis via Multimodal Conditioning
Referring Video Object Segmentation / Multi-Object Tracking Repo
Code used in "Understanding Dimensional Collapse in Contrastive Self-supervised Learning" paper.
CLIP-GEN: Language-Free Training of a Text-to-Image Generator with CLIP
Official PyTorch implementation of "Large-scale Bilingual Language-Image Contrastive Learning" (ICLRW 2022)
PyTorch implementation of a 1.3B text-to-image generation model trained on 14 million image-text pairs
L-Verse: Bidirectional Generation Between Image and Text
Reference models and tools for Cloud TPUs.
[ICCV 2021- Oral] Official PyTorch implementation for Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers, a novel method to visualize any Transformer-…
This is the repo for my experiments with StyleGAN2. There are many like it, but this one is mine. Contains code for the paper Audio-reactive Latent Interpolations with StyleGAN.
DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Models (ICCV 2023)
MAGMA - a GPT-style multimodal model that can understand any combination of images and language. NOTE: The freely available model from this repo is only a demo. For the latest multimodal and multil…
PyTorch code for "Unifying Vision-and-Language Tasks via Text Generation" (ICML 2021)
[ICCV2021 & TPAMI2023] Vision-Language Transformer and Query Generation for Referring Segmentation
Code release for SLIP Self-supervision meets Language-Image Pre-training
CLIP (Contrastive Language-Image Pretraining), Predict the most relevant text snippet given an image
A concise but complete implementation of CLIP with various experimental improvements from recent papers
Repository for "Generating images from caption and vice versa via CLIP-Guided Generative Latent Space Search"
A pytorch Implementation of Open Vocabulary Object Detection with Pseudo Bounding-Box Labels
Open-Vocabulary Instance Segmentation via Robust Cross-Modal Pseudo-Labeling @ CVPR22
Research code for ECCV 2020 paper "UNITER: UNiversal Image-TExt Representation Learning"
official code repo for paper "CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformers"
WIT (Wikipedia-based Image Text) Dataset is a large multimodal multilingual dataset comprising 37M+ image-text sets with 11M+ unique images across 100+ languages.
Official implementation for "Blended Diffusion for Text-driven Editing of Natural Images" [CVPR 2022]
Pytorch implementation of Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors
[CVPR2022 oral] A Simple and Effective Baseline for Text-to-Image Synthesis