Tutorial notebook for applying PEFT with DNA Language models

[rahulbshrestha]

Feature request

A notebook tutorial that shows an example of how to use parameter-efficient fine-tuning techniques from the PEFT library (e.g LoRA, IA3, etc.) with a DNA Language Model (e.g https://huggingface.co/gagneurlab/SpeciesLM).

The tutorial will contain an example of how to load and apply a DNA language model with and without PEFTs on an arbitrary task. There will be detailed descriptions for each part of the code, similar to a high-quality blog post like this.

Motivation

This will show the possibility of using Hugging Face's PEFT library in a computational biological setting e.g with DNA Language Models to solve a task.

Your contribution

I'll submit a PR if this is a good idea.

Question: Should I place the notebook in the examples folder?

[BenjaminBossan]

Thanks for suggesting to provide such a notebook. It would be indeed a good addition for our examples in the examples/ directory.

Note that it looks like people had issues with ESM2 in the past (#1023). If you found a solution, that would be really helpful.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

[BenjaminBossan]

Resolved by #1873