A pipeline adding MiniGPT-4 support to text-generation-webui.
Clone this repo into extensions/multimodal/pipelines directory in text-generation-webui, and install requirements.txt.
This module provides 2 pipelines:
minigpt4-13b- for use with Vicuna-v0-13B LLMminigpt4-7b- for use with Vicuna-v0-7B LLM
To use it in webui, select the appropriate LLM and run server.py with --extensions multimodal --multimodal-pipeline minigpt4-13b (or minigpt4-7b)
The supported parameter combinations for both the vision model, and the projector are: CUDA/32bit, CUDA/16bit, CPU/32bit
Almost all the code in minigpt4 directory is taken from the original MiniGPT4 repo, it was then cleaned up to leave mostly the parts, which are needed for inference. The only modifications are to minigpt4/mini_gpt4.py, but again, they are mostly removing not-needed parts of the code.
In short: I copied enough code from MiniGPT-4, so that inference works(but only for image embeds), then I added a pipeline descriptor.
This is not production-ready code, I take no liability whatsoever, and don't provide any warranty, nor support. Use it only for fun/research