How to train SAEs on my own model？

[likangk]

Hi,
Thank you for your excellent work on the SAE package. I am interested in training Sparse Autoencoders (SAEs) on my own model(LLM in Genomics), which is not included in TransformerLens. Could you please provide some guidance on how to get started with this? Any advice or suggestions would be greatly appreciated.

Thank you for your time and assistance.

Best regards,

[jbloomAus]

This currently isn't supported but could be supported soon. I think the key requirements are:

• Turn a non-T-lens model into a HookedRootModule (a t-lens class).
• Enabling SAE training to work on a local model not imported from T-lens.