EleutherAI/pythia-800m tokenizer adds unusual kwargs, causes a ValueError when evaluating model

[ejmichaud]

It seems like the EleutherAI/pythia-800m tokenizer includes 'token_type_ids' values, but these lead to a ValueError when evaluating the following code:

from transformers import GPTNeoXForCausalLM, AutoTokenizer

model = GPTNeoXForCausalLM.from_pretrained(
  "EleutherAI/pythia-800m",
  revision="step143000",
  cache_dir=".pythia-800m/step143000",
)

tokenizer = AutoTokenizer.from_pretrained(
  "EleutherAI/pythia-800m",
  revision="step143000",
  cache_dir="./pythia-800m/step143000",
)

inputs = tokenizer("Hello, I am", return_tensors="pt")
model.generate(**inputs)

Here is the stack trace:

Traceback (most recent call last):
  File "eval.py", line 76, in <module>
    outputs = model.generate(**inputs, temperature=0.0, max_new_tokens=40)
  File "/om2/user/ericjm/miniconda3/envs/phase-changes/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/om2/user/ericjm/miniconda3/envs/phase-changes/lib/python3.8/site-packages/transformers/generation/utils.py", line 1296, in generate
    self._validate_model_kwargs(model_kwargs.copy())
  File "/om2/user/ericjm/miniconda3/envs/phase-changes/lib/python3.8/site-packages/transformers/generation/utils.py", line 993, in _validate_model_kwargs
    raise ValueError(
ValueError: The following `model_kwargs` are not used by the model: ['token_type_ids'] (note: typos in the generate arguments will also show up in this list)

I can get around this error by simply using a tokenizer from another one of the models. This tokenizer, for instance, works:

tokenizer = AutoTokenizer.from_pretrained(
  "EleutherAI/pythia-19m",
  revision="step143000",
  cache_dir="./pythia-19m/step143000",
)

It seems like the tokenizers are the same for all the models, so this issue is pretty easy to get around, but I just thought I'd report it.

[haileyschoelkopf]

Darn, I thought I'd caught all of these. I'll fix this tomorrow, thanks for raising the issue! :)

Yup, any other tokenizer'd work!

[haileyschoelkopf]

Patched for all 800m intermediate ckpts!