2 Pull requests opened by 2 people

• add qwen2 support for pretraining and finetuning

  #1573 opened Jun 25, 2024

• automatically added to python module search path

  #1584 opened Jul 1, 2024

4 Issues closed by 4 people

• Having issues while merging LoRA attention weights

  #1567 closed Jul 3, 2024

• [Questions] "non_lora_trainables.bin" has an impact on the model

  #1566 closed Jun 19, 2024

• Nothing

  #1539 closed Jun 18, 2024

• [Usage] Multi-GPU training hangs: Watchdog caught collective operation timeout

  #447 closed Jun 11, 2024

35 Issues opened by 34 people

• [Question]

  #1593 opened Jul 6, 2024

• [Usage] After fine-tuning LLaVA 1.5, mm_projector.bin file is not available

  #1592 opened Jul 5, 2024

• LLava always speaks of 2 images

  #1591 opened Jul 3, 2024

• [Question] For image_aspect_ratio, what is the difference between pad, square and anyres

  #1590 opened Jul 3, 2024

• [Question] I got stuck here while doing fine-tuning training.

  #1589 opened Jul 3, 2024

• [Usage] Different seeds are giving the exact same loss when running full finetuning with deepspeed Zero 1,2 or 3

  #1588 opened Jul 2, 2024

• "Argo Tunnel error" when using demo via https://llava.hliu.cc/ => demo is not working

  #1587 opened Jul 1, 2024

• [Question] After merging, not able to infer from the model llava-mistral-v1.6-7b

  #1586 opened Jul 1, 2024

• How to fine tune Lora without images……

  #1585 opened Jul 1, 2024

• Deepspeed Assertion Error after training is completed while saving check points

  #1583 opened Jun 30, 2024

• device mis-match error on pre-training

  #1582 opened Jun 30, 2024

• [Question] lora fine-tuning inference

  #1581 opened Jun 28, 2024

• [Question] Mac run but get importlib.metadata.PackageNotFoundError: No package metadata was found for bitsandbytes

  #1580 opened Jun 27, 2024

• [Usage] Non-deterministic output during inference, occur specifically in the CLIP ViT encoders

  #1579 opened Jun 26, 2024

• [Question] Why LLaVA-1.5-13b keep repeatting answer in the first round conversation?

  #1578 opened Jun 26, 2024

• [Question] LLaVA-NeXT-Video 7B with stride=1 Memory Error

  #1577 opened Jun 26, 2024

• [Question] When I added some code like 'print' in train.py and then run 'finetune_task_lora.sh', it would get the same results of the first run.

  #1576 opened Jun 26, 2024

• [Usage] Can't run inference on multiple GPUs

  #1575 opened Jun 25, 2024

• Smaller llm and/or vision models

  #1574 opened Jun 25, 2024

• ModuleNotFoundError: No module named 'llava'

  #1571 opened Jun 22, 2024

• [Usage] 7B Inferece CUDA Out of Memory for RTX 4090 24GB VRAM

  #1570 opened Jun 21, 2024

• Why use plain text sharegpt datasets for instruction tuning

  #1569 opened Jun 20, 2024

• "Assertion `srcIndex < srcSelectDimSize` failed" in Docker on some systems

  #1568 opened Jun 18, 2024

• [Question] Question on `image_newline` for single image

  #1565 opened Jun 17, 2024

• [Question] What is the accuracy of InstrumentBLIP on VQAv2?

  #1564 opened Jun 17, 2024

• Does the llavaNext 1.5 stage still trained only proj only?

  #1563 opened Jun 17, 2024

• LLaVA Context-length

  #1562 opened Jun 16, 2024

• [Usage] Missing File in Release LLaVA-v1.6 and RuntimeError in model_vqa_qbench.py

  #1561 opened Jun 16, 2024

• [Question] Fine-tune only the projector?

  #1560 opened Jun 16, 2024

• Realtime Image Description

  #1559 opened Jun 14, 2024

• [版本冲突]环境配置要求版本冲突

  #1558 opened Jun 13, 2024

• [Question]About Full Parameter Finetune cost ？

  #1557 opened Jun 13, 2024

• [Discussion] why are the results on post inconsistent with opencompass?

  #1556 opened Jun 12, 2024

• [Usage] Not able to fine tune the LLaVA model with llava-v1.5-7b.

  #1554 opened Jun 11, 2024

• [Usage] ValueError: The current `device_map` had weights offloaded to the disk.

  #1553 opened Jun 10, 2024

47 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• process multi images

  #1292 commented on Jul 8, 2024 • 12 new comments

• [Usage] Missing "trainer_state.json" when resuming training from saved checkpoints

  #1164 commented on Jun 30, 2024 • 7 new comments

• How to fine-tune llava-v1.6-mistral-7b on GQA dataset

  #1544 commented on Jul 2, 2024 • 6 new comments

• [Usage] None of the inputs have requires_grad=True. Gradients will be None

  #1475 commented on Jul 6, 2024 • 6 new comments

• [Question] Training with Qwen2 backend got loss 0

  #1153 commented on Jun 24, 2024 • 6 new comments

• [Question] No matter what I ask, my model always gives the same answer.

  #1546 commented on Jul 3, 2024 • 4 new comments

• [Usage] deploy a llava model on AWS SageMaker use HuggingFaceModel

  #1312 commented on Jun 12, 2024 • 2 new comments

• [Usage] Error in load liuhaotian/llava-v1.6-34b checkpoint.

  #1386 commented on Jul 9, 2024 • 2 new comments

• Could we have a model with DINOv2 as image encoder

  #597 commented on Jun 21, 2024 • 2 new comments

• ERROR: Could not consume arg: val

  #778 commented on Jun 16, 2024 • 2 new comments

• [Usage] Merging LoRa weights into llava-13b fails with bizarre error

  #1552 commented on Jul 5, 2024 • 2 new comments

• [Question] Getting output likelihood scores from the model

  #108 commented on Jul 2, 2024 • 2 new comments

• Does vistion tower trained during starge 2 (Visual Instruction Tuning)?

  #1537 commented on Jun 19, 2024 • 2 new comments

• [Usage] Unable to load LLaVA v1.6 models

  #1039 commented on Jun 28, 2024 • 1 new comment

• [Usage] Not able to import llava file while fine tunning

  #1318 commented on Jun 27, 2024 • 1 new comment

• [Usage] 我在一个A100 80GB下进行全模型训练，告诉我显存不够

  #827 commented on Jul 2, 2024 • 1 new comment

• [Usage] Error while using finetuned model

  #1519 commented on Jun 9, 2024 • 1 new comment

• Question about multiple images in the same prompt

  #1479 commented on Jul 3, 2024 • 1 new comment

• [Usage] Continue training from pre-trained checkpoint

  #1496 commented on Jul 4, 2024 • 1 new comment

• [Question] Reproduce LLaVA with Mistral backened

  #1435 commented on Jul 5, 2024 • 1 new comment

• Cant start controller for LLaVa "llava' is already used by a Transformers config, pick another name"

  #974 commented on Jul 5, 2024 • 1 new comment

• [Question] LLaVA-1.6 Training Data

  #1353 commented on Jul 7, 2024 • 1 new comment

• [Question] RuntimeError: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when calling `cublasGemmEx( handle, opa, opb, m, n, k, &falpha, a, CUDA_R_16F, lda, b, CUDA_R_16F, ldb, &fbeta, c, CUDA_R_16F, ldc, CUDA_R_32F, CUBLAS_GEMM_DFALT_TENSOR_OP)`

  #938 commented on Jul 8, 2024 • 1 new comment

• [Usage] Getting IndexErrors finetuning on a custom dataset

  #677 commented on Jul 8, 2024 • 1 new comment

• [Usage] Is there a way to describe one image and the next?

  #605 commented on Jul 8, 2024 • 1 new comment

• Add NPU support for Llava

  #1446 commented on Jun 13, 2024 • 1 new comment

• [Question] no mm_projector.bin after finetune.sh

  #852 commented on Jun 9, 2024 • 1 new comment

• [Usage] How can I implemet few shot learning on LLaVa

  #1202 commented on Jun 10, 2024 • 1 new comment

• repetition penalty

  #836 commented on Jun 11, 2024 • 1 new comment

• [BUG] ../aten/src/ATen/native/cuda/Indexing.cu:1146: indexSelectLargeIndex: block: [375,0,0], thread: [127,0,0] Assertion `srcIndex < srcSelectDimSize` failed.

  #1129 commented on Jun 12, 2024 • 1 new comment

• [Question] LLaVA 1.5 sizes and vision encoder

  #1083 commented on Jun 12, 2024 • 1 new comment

• [Usage] Cannot load the tuned projector weights

  #474 commented on Jun 12, 2024 • 1 new comment

• What's the minimum GPU requirement to Fine Tune LLaVA

  #944 commented on Jun 13, 2024 • 1 new comment

• [Question] Regarding Captioning Evaluation on Flickr30k

  #768 commented on Jun 13, 2024 • 1 new comment

• How to test after finetuning the LLaVA-v1_5-7b

  #1111 commented on Jun 16, 2024 • 1 new comment

• [Usage] When will the eval script for llava v1.6 be available?

  #1326 commented on Jun 17, 2024 • 1 new comment

• [Question] Have you finished LoRA/QLoRA training?

  #206 commented on Jun 18, 2024 • 1 new comment

• [Question] How to use in-context learning in LLaVA?

  #1357 commented on Jun 18, 2024 • 1 new comment

• errors in MME evaluation

  #1527 commented on Jun 19, 2024 • 1 new comment

• [Usage] can‘t run gradio_web_server.py

  #986 commented on Jun 19, 2024 • 1 new comment

• [Question] Cannot reproduce MME results on LLaVA-1.5-7B

  #630 commented on Jun 20, 2024 • 1 new comment

• [Question] ImportError: cannot import name 'LlavaLlamaForCausalLM' from 'llava.model' (/root/LLaVA/llava/model/__init__.py)

  #1208 commented on Jun 21, 2024 • 1 new comment

• [Question] Why apply the causal mask to image tokens in the attention operations of LLM?

  #1529 commented on Jun 22, 2024 • 1 new comment

• load_pretrained_model() got an unexpected keyword argument 'offload_folder'

  #1212 commented on Jun 25, 2024 • 1 new comment

• [Question] I finetuning llava1.5-7b model using lora on llava_mix_665k dataset but get 545 on MME perception score. Has anyone else encountered this issue?

  #1461 commented on Jun 25, 2024 • 1 new comment

• [Usage] The deterministic mode did not set in eval_model() function

  #1013 commented on Jun 26, 2024 • 1 new comment

• Update pyproject.toml for lora dependencies;

  #1439 commented on Jul 3, 2024 • 0 new comments