The DeepSpeed unit test suite is very good, and I suggest that whoever picks this up use them as a template.

Once we have a solid test suite, they should be applied to every PR as a github action, similar to how DeepSpeed does it.

@StellaAthena @Quentin-Anthony I can pick this up. I'll need a day or two to go through the codebase as well as the DeepSpeed unit test suite. Will keep this thread updated.

@whiz-Tuhin That's awesome, welcome to the team! I went ahead and edited the OP with some additional things that should probably be incorporated though I'm sure it's still not totally comprehensive. That said, don't get overwhelmed by the number of tests to write. A small and reliable test suite that doesn't cover every feature but does cover major ones would be a huge value add. We can then build on that over time to be more and more comprehensive.

Quentin knows what he's talking about, so I would definitely start by working on porting the DeepSpeed tests over, removing the stuff we don't need and adding tests for things DeepSpeed doesn't support.

I've sent you an invite to the EleutherAI Org that will allow you to work on a non-main branch without having to mess around with forking the library.

Looking again for someone to pick this up!

Hi @Quentin-Anthony @StellaAthena - just checking if you're still looking for a volunteer here? I have good bandwidth to work on this over the next 2 weeks, which I think should be more than enough time (& I've been looking for an opportunity to contribue for a while!).

Hi @Quentin-Anthony @StellaAthena - just checking if you're still looking for a volunteer here? I have good bandwidth to work on this over the next 2 weeks, which I think should be more than enough time (& I've been looking for an opportunity to contribue for a while!).

Yes! That would be phenomenal

Hi @Quentin-Anthony @StellaAthena - just checking if you're still looking for a volunteer here? I have good bandwidth to work on this over the next 2 weeks, which I think should be more than enough time (& I've been looking for an opportunity to contribue for a while!).

@mkerin -- Great to hear! Please reach out to me here or over Discord if you need any help or have any questions. A good first step would be to just run our existing CPU tests locally, then start with some of the simple model option tests.

Hey Quentin - thank you for the tips!

I have got the existing CPU tests running & started working on integration tests for the data processing functionality (I didn’t see your message about focussing on model options until just now).

I have dropped you a line on discord. My handle there is mkez.