Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ggml : storing strides as number of elements instead of number of bytes #623

Open
slaren opened this issue Nov 27, 2023 · 1 comment
Open
Labels
refactoring Refactoring

Comments

@slaren
Copy link
Collaborator

slaren commented Nov 27, 2023

Currently, we store the strides between elements of each dimension as a number of bytes in ggml_tensor::nb. In practice, this complicates code because strides always need to be multiplied by the element size, and accessing elements requires first casting the pointers to char *.

I am not sure if there are any cases where we would want a byte stride that isn't a multiple of the element size, as this would mean that the addresses would no longer be aligned to the element size, which is not ok in many platforms. Therefore I think we could simplify the code a bit by storing strides as numbers of elements instead of numbers of bytes.

@ggerganov
Copy link
Owner

Yup, I guess it would be an improvement. Probably we can rename nb to ns so that we get errors in all places in the codebase when refactoring this. And leave a comment explaining to 3rd party devs how to update if they are using nb somewhere in their projects

@ggerganov ggerganov added the refactoring Refactoring label Nov 27, 2023
@ggerganov ggerganov changed the title Storing strides as number of elements instead of number of bytes ggml : storing strides as number of elements instead of number of bytes Nov 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
refactoring Refactoring
Projects
Status: Todo
Development

No branches or pull requests

2 participants