Paper | Blog | Model Checkpoint Download | HuggingFace
This repository contains artifacts for the Meta Chameleon model from FAIR, Meta AI. In this repository is:
- Standalone Inference Code — a fast GPU-based inference implementation for running model checkpoints
- Input-Output Viewing — a harness for richly viewing multimodal model inputs and outputs with a browser-based tool
- Evaluation Prompts — mixed-modal and text-only prompts for human evaluation
Running constituent components for inference and the input-output viewer currently requires a CUDA-capable GPU. If you'd like to run inference on other hardware, other inference implementations, including HuggingFace, are platform agnostic.
First, pip install this repository:
pip install -U git+https://github.com/facebookresearch/chameleon.git
Alternatively, if you want access to the full visualizer, you'll need to clone this repository (instead of installing), then pip install from the repository root:
git clone https://github.com/facebookresearch/chameleon.git
cd chameleon
pip install -e .
Model checkpoints and configs must be downloaded before running inference or the viewer. After requesting model access, run the following script, adding pre-signed download URL you were emailed when prompted:
python -m chameleon.download_data [pre-signed URL]
(you can also paste the command given in the email containing the download link)
The viewer visualizes multi-modal model input and output. It is most easily run with docker-compose
. You'll need to clone the repository, not just a pip install.
The following runs both the service and viewer interface.
By default, this runs the 7B parameter model. You can change the
model_path
variable in./config/model_viewer.yaml
to select another model and alter other configuration:
docker-compose up --build
You can open the viewer at https://localhost:7654/
The miniviewer is a light weight debug visualizer, that can be run with:
python -m chameleon.miniviewer
This runs the 7B parameter model. To run the 30B model, use the following command:
python -m chameleon.miniviewer --model-size 30b
You can open the miniviewer at https://localhost:5000/
.
Use of this repository and related resources are governed by the Chameleon Research License and the LICENSE file.
To cite the paper, model, or software, please use the below:
@article{Chameleon_Team_Chameleon_Mixed-Modal_Early-Fusion_2024,
author = {Chameleon Team},
doi = {10.48550/arXiv.2405.09818},
journal = {arXiv preprint arXiv:2405.09818},
title = {Chameleon: Mixed-Modal Early-Fusion Foundation Models},
url = {https://github.com/facebookresearch/chameleon},
year = {2024}
}