[Questions] "non_lora_trainables.bin" has an impact on the model

[supech]

Describe the issue

I originally thought that the output of lora training ouput "non_lora_trainables.bin" could be shared with other checkpoints as the literal meaning.
So I did a test. I simply trained it twice. Only the "non_lora_trainables.bin" trained that time will have normal image dialogue output. If "non_lora_trainables.bin" is exchanged, it will be abnormal.
I would like to ask if there is a way to save "non_lora_trainables.bin" and "config.json" in all checkpoints during the lora training process. Otherwise, the checkpoints are saved but cannot be used and it is difficult to correctly judge the model performance.

[supech]

#729
#844

The discussions in these two issues are useful to me.