Efficient Multimodal Large Language Models: A Survey

Official repository for "Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing". Your efficient and high-quality synthetic data generation pipeline!

Official pytorch implementation of "RITUAL: Random Image Transformations as a Universal Anti-hallucination Lever in LVLMs"

This is the first released survey paper on hallucinations of large vision-language models (LVLMs). To keep track of this field and continuously update our survey, we maintain this repository of rel…

This is the official repo for Debiasing Large Visual Language Models, including a Post-Hoc debias method and Visual Debias Decoding strategy.

Anole: An Open, Autoregressive and Native Multimodal Models for Interleaved Image-Text Generation

[CVPR'2024 Highlight] Official PyTorch implementation of the paper "VTimeLLM: Empower LLM to Grasp Video Moments".

SimPO: Simple Preference Optimization with a Reference-Free Reward

RLAIF-V: Aligning MLLMs through Open-Source AI Feedback for Super GPT-4V Trustworthiness

[ACL 2024 🔥] Video-ChatGPT is a video conversation model capable of generating meaningful conversation about videos. It combines the capabilities of LLMs with a pretrained visual encoder adapted fo…

Explainability for Vision Transformers

A RLHF Infrastructure for Vision-Language Models

LLMs build upon Evol Insturct: WizardLM, WizardCoder, WizardMath

When do we not need larger vision models?

Benchmarking large language models' complex reasoning ability with chain-of-thought prompting

Official repo for "Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models"

Dense Connector for MLLMs

GPT4V-level open-source multi-modal model based on Llama3-8B

CuMo: Scaling Multimodal LLM with Co-Upcycled Mixture-of-Experts

📖 A curated list of resources dedicated to hallucination of multimodal large language models (MLLM).

🙌 OpenHands: Code Less, Make More

DeepSeek-VL: Towards Real-World Vision-Language Understanding

Reading notes about Multimodal Large Language Models, Large Language Models, and Diffusion Models

Mixture-of-Experts for Large Vision-Language Models

🔥🔥 LLaVA++: Extending LLaVA with Phi-3 and LLaMA-3 (LLaVA LLaMA-3, LLaVA Phi-3)

Accelerating the development of large multimodal models (LMMs) with lmms-eval

A Unified Pixel-level Vision LLM for Understanding, Generating, Segmenting, Editing

A family of lightweight multimodal models.

Code for the paper "Evaluating Large Language Models Trained on Code"