-
Notifications
You must be signed in to change notification settings - Fork 22k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
No factory functions for strided quantized tensors #74540
Comments
Note that pytorch/ao#292 means that not all strides are valid; the innermost dimension must be kept contiguous |
Quantization pw support differs from eager a lot (e.g. they don't support arbitrary broadcast), and they typically offload to 3rd party library rather than rely on TI, so just enabling empty_strided won't solve this |
per discussion with @vkuzo there is not a good way to use TensorIterator on quantized tensors and preserve strides, and set this issue as low priority. |
Is there any news in this front? Thanks |
How would one convert and empty strided tensor to a strided tensor? |
it seems that |
Hello, I'm trying to quantize yolov8, this error redirects me here. Error is located here:
and the logs: empty_strided not supported on quantized tensors yet see #74540 File "/usr/local/lib/python3.8/dist-packages/ultralytics/nn/modules/block.py", line 341, in forward |
I solved this using torch.nn.quantized.FloatFunctional(). |
@ezyang Hi! I am using pytorch 2.0.0 and I am also facing this issue:
|
Has this problem been solved and can you share the solution? @ezyang |
The old school quantization tensors are being phased out in favor of https://github.com/pytorch/ao/, I recommend taking a look there. |
I ran into this issue while trying to static quant an open_clip model (from HF download and init). I sort of did a bit of workaround in CLIPVisionEmbeddings's forward method:
for whatever reasons, that quantized conv2d ("patch_embedding")layer from torch.ao.nn.quantized.modules.conv.Conv2d has weight() returning the weight tensor, rather than .weight and the next thing is what brings me to this issue: File ~/vss_env/lib/python3.10/site-packages/transformers/models/clip/modeling_clip.py:188, in CLIPVisionEmbeddings.forward(self, pixel_values) RuntimeError: empty_strided not supported on quantized tensors yet see #74540 |
🐛 Describe the bug
For non-quantized tensors, there is both
empty
andempty_strided
. However, for quantized tensors there are onlyempty
variants for functions. This means that it is difficult for quantized operators to properly preserve strides when they should (e.g., TensorIterator style ops).If this is affecting you please comment here.
Versions
master
cc @jerryzh168 @jianyuh @raghuramank100 @jamesr66a @vkuzo
The text was updated successfully, but these errors were encountered: