Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Why add 1024*1024 to ctx_size #387

Closed
Metal-joker opened this issue Jul 14, 2023 · 2 comments
Closed

Why add 1024*1024 to ctx_size #387

Metal-joker opened this issue Jul 14, 2023 · 2 comments

Comments

@Metal-joker
Copy link

struct ggml_init_params params = {
/*.mem_size =*/ ctx_size + 1024*1024,
/*.mem_buffer =*/ NULL,
/*.no_alloc =*/ false,
};

new to cpp and ggml, could anyone help me understand this?

@ggerganov
Copy link
Owner

It should be n_tenors * ggml_tensor_overhead() instead of 1024*1024.
But I was just being lazy to count the number of tensor and decided to put a some extra space that should be enough

@Metal-joker
Copy link
Author

It should be n_tenors * ggml_tensor_overhead() instead of 1024*1024. But I was just being lazy to count the number of tensor and decided to put a some extra space that should be enough

Thanks man, got it now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants