Possible to train LoRa on mac m2 GPU ?

[Anvil-Late]

Hi,

I want to finetune the wizard-vicuna-7b model (currently using HF because GGML seems to be broken for now) and to add some medical knowledge.
So far my dataset is ready, loads fine and has the proper structure, and the training works as well but is a bit slow.
I'd like to use the GPU to train my lora faster ; is there any way to do that with the text-generation-webui ?

I'm using a mac m2 pro with 32G of unified memory.

PS : I looked around and didn't find a similar issue but if the question has already been asked, don't hesitate to link it to me !

[Anvil-Late]

Closing my own issue : apparently work is underway on llama.cpp to integrate fine-tuning, including on mac gpu.

Source : ggerganov/ggml#8

[QueryType]

Hmm.. it is a pity you closed the issue. As far as we know, finetuning on llama.cpp is still a bit shaky (at least for me) on the mac m1/m2. Also as per ggerganov, finetuning on metal, is low prior item.