Missing non-LoRA key tok_embeddings.weight from base model dict

[vasicvuk]

When running following commands:

  File "/opt/Projects/torchtune/venv/lib/python3.10/site-packages/recipes/lora_finetune_single_device.py", line 302, in _setup_model
    validate_missing_and_unexpected_for_lora(
  File "/opt/Projects/torchtune/venv/lib/python3.10/site-packages/torchtune/modules/peft/peft_utils.py", line 334, in validate_missing_and_unexpected_for_lora
    raise AssertionError(f"Missing non-LoRA key {k} from base model dict")
AssertionError: Missing non-LoRA key tok_embeddings.weight from base model dict

I am getting error:

AssertionError: Missing non-LoRA key tok_embeddings.weight from base model dict

Is there something I am doing wrong?

[RdoubleA]

Hi @vasicvuk, which model are you using? Do you mind sharing your config?

[vasicvuk]

@RdoubleA phi3 with the config from the repo, I guess there is some extra steps about weights to do for phi3?