Evaluation for MegatronT5 Model

[wangyanbao666]

Hi I have noticed that there are existing support for some nemo models. But it does not seem that there is a support for MegatronT5 Model. Anyone has ideas how to evaluate this model?

[haileyschoelkopf]

Hi! We don't currently support this, though would welcome contributions adding it.

Perhaps @sergiopperez has input on how simple it would be to extend the current NeMo integration into Megatron-T5?

[wangyanbao666]

Hi any updates on how to evaluate T5? Could you please give some clue on how to extend to Megatron-T5?

[StellaAthena]

Hi any updates on how to evaluate T5? Could you please give some clue on how to extend to Megatron-T5?

If you are interested in evaluating a pretrained model such as NeMo Megatron T5 that has been released on Hugging Face, the Hugging Face version is compatible with this library. However we still lack support for models native to the Megatron library.

[haileyschoelkopf]

Closing as NeMo Megatron-T5 support won't be prioritized, though we'd welcome a contribution for it.