Smoke tests with tiny gpt2, fix CCSReporter #149
Conversation
thejaminator
changed the title
| but you'll need to make deberta fp32 instead of fp16 | ||
| because pytorch cpu doesn't support fp16 | ||
| """ | ||
|
|
There was a problem hiding this comment.
basically the deberta model itself does some fp16 operations that aren't supported. but tiny-gpt doesn't. so that why tiny-gpt works.
i'll try to figure that out in another MR. i think we could convert fp16 models to fp32 if its CPU, but seems hacky?
There was a problem hiding this comment.
I actually don't view this as hacky. In general float16 isn't supported on CPUs. We could check the whether we're running on CPU right when we call AutoModel.from_pretrained and tell HF to always load the model as float32 in that case, and use the dtype of the checkpoint otherwise.
There was a problem hiding this comment.
^ just made this change, so if you want to add a deberta test you can. I think tiny gpt2 is probably sufficient for now though
| u[:] = torch.einsum("...ij,...j->...i", A, V[..., k, :]) | ||
|
|
||
| RuntimeError: select(): index 1 out of range for tensor of size [1, 2] | ||
| at dimension 0 |
There was a problem hiding this comment.
EigenReporterConfig is broken for tiny-gpt2. didn't investigate, would like to push the fix for CCS first
There was a problem hiding this comment.
Thanks for noticing this, I just pushed a fix
thejaminator
changed the title
This MR adds a smoke test for CCSReporter with tinygpt2
Subsequent issues: