Fixes for higher sequence lenghts

[ruanslv]

Simple changes just extending switches and asserts

[ngoyal2707]

is it okay with you if we merge this directly to v3? and we can push the small PR to metaseq to make it work with v3?

[ruanslv]

@ngoyal2707 would you be ok with a merge here too? Your metaseq PR (https://github.com/facebookresearch/metaseq/pull/300/commits) as a whole makes it possible to use fairseq_v3, but the Namespace change alone is not enough in my local tests. I've tried a few different things but couldn't figure out what else exactly from the metaseq branch I'd need to include.

Given you are working on merging your changes anyway, I thought best option would be: keep fairseq_v2 for now for seq_len tests, and when your PR is fully merged we go back to fairseq_v3.

Stack trace of error I'm getting: P528724412

Other things I've tried and didn't work:

• using your metaseq-internal branch: https://github.com/fairinternal/metaseq-internal/compare/main...ngoyal_sequence_parallel_options
• reverting metaseq back to the main where you PR commits are written on top of and adding only namespace change to it.

Command I'm using to test:
$ python -m metaseq_internal.fb_sweep.sweep_openlm_baselines -n 1 -g 8 -t 1 -p test_fv3 --azure --local --model-size 125m --checkpoints-dir /shared/home/ruanrms/checkpoints/test_fv3/ --no-wandb