[CVPR 2024] Official implementation of "ViTamin: Designing Scalable Vision Models in the Vision-language Era"

Official Repository of ChartX & ChartVLM: A Versatile Benchmark and Foundation Model for Complicated Chart Reasoning

Schedule-Free Optimization in PyTorch

Awesome multilingual OCR toolkits based on PaddlePaddle (practical ultra lightweight OCR system, support 80+ languages recognition, provide data annotation and synthesis tools, support training and…

Monitor Memory usage of Python code

Set-of-Mark Prompting for LMMs

COYO-700M: Large-scale Image-Text Pair Dataset

make a book from imessages

Code used for the creation of OBELICS, an open, massive and curated collection of interleaved image-text web documents, containing 141M documents, 115B text tokens and 353M images.

MultimodalC4 is a multimodal extension of c4 that interleaves millions of images with text.

Easily compute clip embeddings and build a clip retrieval system with them

Open-sourced codes for MiniGPT-4 and MiniGPT-v2 (https://minigpt-4.github.io, https://minigpt-v2.github.io/)

GuessWhat?! Baselines

[CVPR 2024 🔥] Grounding Large Multimodal Model (GLaMM), the first-of-its-kind model capable of generating natural language responses that are seamlessly integrated with object segmentation masks.

a state-of-the-art-level open visual language model | 多模态预训练模型

Official Implementation of Paella https://arxiv.org/abs/2211.07292v2

Implementation of Generating Diverse High-Fidelity Images with VQ-VAE-2 in PyTorch

PyTorch package for the discrete VAE used for DALL·E.

Magicoder: Source Code Is All You Need

[ICLR 2024] Evaluating Large Language Models at Evaluating Instruction Following

[ACL2023] We introduce LLM-Blender, an innovative ensembling framework to attain consistently superior performance by leveraging the diverse strengths of multiple open-source LLMs. LLM-Blender cut …

Reference implementation for DPO (Direct Preference Optimization)

Drop in a screenshot and convert it to clean code (HTML/Tailwind/React/Vue)

Yet Another Python Profiler, but this time multithreading, asyncio and gevent aware.

Official repository of NEFTune: Noisy Embeddings Improves Instruction Finetuning

Simple real time visualisation of the execution of a Python program.

A simulation framework for RLHF and alternatives. Develop your RLHF method without collecting human data.