Skip to content

Commit

Permalink
Update LLaVA-NeXT.md
Browse files Browse the repository at this point in the history
  • Loading branch information
kcz358 committed May 9, 2024
1 parent 762e622 commit bbbd746
Showing 1 changed file with 0 additions and 41 deletions.
41 changes: 0 additions & 41 deletions docs/LLaVA-NeXT.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,47 +10,6 @@ pip install git+https://github.com/EvolvingLMMs-Lab/lmms-eval.git

### Quick Start With HuggingFace

```Python
from llava.model.builder import load_pretrained_model
from llava.mm_utils import get_model_name_from_path
from llava.eval.run_llava import eval_model

model_path = "liuhaotian/llava-v1.5-7b"

tokenizer, model, image_processor, context_len = load_pretrained_model(
model_path=model_path,
model_base=None,
model_name=get_model_name_from_path(model_path)
)
```

Check out the details wth the `load_pretrained_model` function in `llava/model/builder.py`.

You can also use the `eval_model` function in `llava/eval/run_llava.py` to get the output easily. By doing so, you can use this code on Colab directly after downloading this repository.

``` python
model_path = "liuhaotian/llava-v1.5-7b"
prompt = "What are the things I should be cautious about when I visit here?"
image_file = "https://llava-vl.github.io/static/images/view.jpg"

args = type('Args', (), {
"model_path": model_path,
"model_base": None,
"model_name": get_model_name_from_path(model_path),
"query": prompt,
"conv_mode": None,
"image_file": image_file,
"sep": ",",
"temperature": 0,
"top_p": None,
"num_beams": 1,
"max_new_tokens": 512
})()

eval_model(args)
```

### Inference code
Here is a quick inference code using [`llavanext-llama3-8B`](https://huggingface.co/lmms-lab/llama3-llava-next-8b) as an example. You will need to install [`flash-attn`](https://github.com/Dao-AILab/flash-attention) to use this code snippet. If you don't want to install it, you can set `attn_implementation=None` when load_pretrained_model
```python
from llava.model.builder import load_pretrained_model
Expand Down

0 comments on commit bbbd746

Please sign in to comment.