-
-
Notifications
You must be signed in to change notification settings - Fork 611
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Restrict model options in transformers examples #2189
Comments
Thanks for reporting that @KickItLikeShika , I think the first option is good enough. Could you please provide few links on the code and explain in details on how to do that. Such that someone new could try to tackle the issue. Thanks |
@vfdev-5 Updated the description. |
@KickItLikeShika Another way I think we can fix this is by passing the dictionary that tokenizer returns instead of making our own dictionary in |
@Ishan-Kumar2 seems like a good idea for resolving the input issues, but we still have the same issue related to the Transformer output, most of other transformers don't have a |
Initially it looks good, what do you think @vfdev-5? if that's OK, can you try this out please? @Ishan-Kumar2 |
OK for me if you think it could be understandable by other users. |
In transformers example https://github.com/pytorch/ignite/tree/master/examples/contrib/transformers it's up to user to override the default model which is
bert-base-uncased
, a lot of models take similar inputs toBERT
and similar outputs too, but models likedistilbert-base-uncased
,distilroberta-base
,bart-base
(and many other models) will not work here as they work on a bit different way regarding to the inputs and outputs of the model, check here for more info: https://huggingface.co/transformers/model_doc/distilbert.html#distilbertmodelSo we will get an error similar to this while using
DistilBERT
And also if we used something like
XLNet
that won't work well and we will have dimensions issue, becauseXLNet
doesn't return apooler_output
, check here https://huggingface.co/transformers/model_doc/xlnet.html#transformers.XLNetModelSo what we should do is:
BERT
,RoBERTa
(and if there is others we mentions them).EDIT: After deciding to go with the first option, what we need to do now is to remove
model
from the argument, and setconfig['model'] = 'bert-base-uncased'
manually insiderun
method, and mention in the docs we are usingBERT
by default, and maybe leave a note about if the user wants to try another model, he should that himself.The text was updated successfully, but these errors were encountered: