[ICLR 2024 (Spotlight)] "Frozen Transformers in Language Models are Effective Visual Encoder Layers"

HistoGym: A Reinforcement Learning Environment for Histopathological Image Analysis

[MICCAI 2024] AdaCBM: An Adaptive Concept Bottleneck Model for Explainable and Accurate Diagnosis

M3D: Advancing 3D Medical Image Analysis with Multi-Modal Large Language Models

BiomedGPT: A Generalist Vision-Language Foundation Model for Diverse Biomedical Tasks

Official repository of OFA (ICML 2022). Paper: OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework

UrFound: Towards Universal Retinal Foundation Models via Knowledge-Guided Masked Modeling

Towards Large Multimodal Models as Visual Foundation Agents

[ACL 2024 🔥] Video-ChatGPT is a video conversation model capable of generating meaningful conversation about videos. It combines the capabilities of LLMs with a pretrained visual encoder adapted fo…

[CVPR2024] ModaVerse: Efficiently Transforming Modalities with LLMs

This is the official repository of our paper "MedTrinity-25M: A Large-scale Multimodal Dataset with Multigranular Annotations for Medicine“

[ECCV 2024🔥] Official implementation of the paper "ST-LLM: Large Language Models Are Effective Temporal Learners"

OphNet: A Large-Scale Video Benchmark for Ophthalmic Surgical Workflow Understanding

Integrated Image-based Deep Learning and Language Models for Primary Diabetes Care

This is a sample solution using the MONET model.

Pre-trained Multimodal Large Language Model Enhances Dermatological Diagnosis using SkinGPT-4

✨✨Latest Advances on Multimodal Large Language Models

Implementing Siamese networks with a contrastive loss for similarity learning

Siamese Network implementation using Pytorch

This repository contains the python code for a Siamese neural network to detect changes in aerial images using Tensorflow.

A generalist foundation model for healthcare capable of handling diverse medical data modalities.

VILA - a multi-image visual language model with training, inference and evaluation recipe, deployable from cloud to edge (Jetson Orin and laptops)

ViLa-MIL: Dual-scale Vision-Language Multiple Instance Learning for Whole Slide Image Classification (CVPR 2024)

Python library for evaluating and enhancing model robustness in medical image analysis.

HEST: Bringing Spatial Transcriptomics and Histopathology together

Cambrian-1 is a family of multimodal LLMs with a vision-centric design.

skin tone analysis for representation in medical educational materials using machine learning