inference error

[tsw123678]

I merge the lora weight and use it for infering with your infer script，but i encounter the error：

(llava) root@bj1oj9u6aucjn-0:/x/tsw/llavapp/LLaVA-pp/LLaVA# python run_llava.py
[2024-05-10 09:15:01,902] [INFO] [real_accelerator.py:161:get_accelerator] Setting ds_accelerator to cuda (auto detect)
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [02:38<00:00, 39.64s/it]
Loading LoRA weights from ./checkpoints/llama3-finetune-lora
Merging weights
Convert to FP16...
Traceback (most recent call last):
File "/x/tsw/llavapp/LLaVA-pp/LLaVA/run_llava.py", line 152, in
eval_model(args)
File "/x/tsw/llavapp/LLaVA-pp/LLaVA/run_llava.py", line 67, in eval_model
if model.config.mm_use_im_start_end:
File "/x/sherlor/envs/llava/lib/python3.10/site-packages/transformers/configuration_utils.py", line 265, in getattribute
return super().getattribute(key)
AttributeError: 'LlamaConfig' object has no attribute 'mm_use_im_start_end'

the config file after merging is to simple

[mmaaz60]

Hi @tsw123678,

Thank you for your interest in our work. Please note the following for running LLaMA-3-V model inference,

1. The transformers version needs to be latest, e.g. pip install git+https://github.com/huggingface/transformers@a98c41798cf6ed99e1ff17e3792d6e06a2ff2ff3
2. You need to use the llama3 chat template.

However, from the error you shared, it looks like something is wrong with the checkpoints. Could you please try if you can run the inference using our provided merged checkpoints at https://huggingface.co/MBZUAI/LLaVA-Meta-Llama-3-8B-Instruct?

Thank You