Transformer Example Update - AutoModelforSequenceClassification

[Ishan-Kumar2]

Fixes #2189

Description:
Changes the model from AutoModel to AutoModelforSequenceClassification so that any model which supports Sequence Classification list can be used without the need to make changes to the code.
Check list:

• New tests are added (if a new feature is added)
• New doc strings: description and/or example code are in RST format
• Documentation is updated (if required)

[sdesrozis]

Tried on Colab and it seems this does not work fine... @Ishan-Kumar2 could you check please ? Thanks !!

[KickItLikeShika]

Thanks for investigating @sdesrozis. Thanks for the PR @Ishan-Kumar2, please make sure your code works before pushing your updates, thanks!

[Ishan-Kumar2]

Hi @sdesrozis, I tested on my machine and on colab just now, its seems to be working for me. Could you share the notebook you running or please have a look at this one(I stopped the run after 1 epoch)?
https://colab.research.google.com/drive/1xJ6tB8GVf_1lZfj3xV3rFwdbwi41R9qW#scrollTo=EIf3dk6PC979
Thanks

[sdesrozis]

@Ishan-Kumar2 I confirm it does not work using GPU which is not enabled in your notebook. Anyway, it would be great that your branch works fine (version 0.5 instead of 0.4.6)

%%shell 

git clone https://github.com/Ishan-Kumar2/ignite.git
cd ignite
git checkout transformers_ex_refactor
python setup.py install
cd examples/contrib/transformers
pip install -r requirements.txt
python main.py run --batch_size=2 --max_length=10

[Ishan-Kumar2]

@sdesrozis thanks for the review, I realized there was a slight mistake in the transfer of the input to GPU. I have fixed it now and tested on Colab, it works fine. Please take a look thanks!

[KickItLikeShika]

Thanks for the updates @Ishan-Kumar2, the first issue has been solved, but there is another issue related to GPU memory in Colab with the default max_length and batch_size, after running th evaluation it crashes because GPU is out of memory, I suggest using gc.collect() + torch.cuda.empty_cache() + being careful with what needs to be deleted/what needs to stay on CPU to avoid this issue, and also I suggest decrease the default batch_size a bit, 128 is too large for a transformer, please fix this issue. Thanks, Ishan!

[KickItLikeShika]

Hey @Ishan-Kumar2, any updates of this?

[Ishan-Kumar2]

@KickItLikeShika, added the commit. Sorry for the delay, I was stuck on why there was a sudden increase in GPU memory usage when going from Epoch 0 to Epoch 1.

[KickItLikeShika]

Very nice work, Ishan! thanks a lot for the updates!

[Ishan-Kumar2]

@KickItLikeShika I have made a minor change reverting the validation dataloader batch_size to 2 * train_loader batch_size. This wasn't the reason for the out of memory earlier.

[Priyansi]

Hey @Ishan-Kumar2 and @KickItLikeShika , thank you for working on this PR. Just wanted to let you know that we are in the process of shifting ignite/examples to https://github.com/pytorch-ignite/examples. While this PR is a great improvement, this might not be relevant and can be removed once we make tutorials out of this, one of which is already in the process: pytorch-ignite/examples#32 which uses AutoModelForSequenceClassification.
Also, you are always welcome to contribute to pytorch-ignite/examples!

[sdesrozis]

@Ishan-Kumar2 Thanks ! LGTM !