Why use plain text sharegpt datasets for instruction tuning

[AshOneN]

Question

ShareGPT is used for instruction fine-tuning, with the aim of inserting data from image independent pure text conversations into multiple rounds of image conversations, so that the model can distinguish which instructions are image independent and which are image related?

[AshOneN]

I found that the plain text dialogue is not inserted into the graphic dialogue, and adding plain text to the MLLM to fine-tune the LLM can improve the MLLM's instruct compliance ability？