ggml : ggml_graph_compute should not require ggml_context
#287
Labels
Comments
CCLDArjun
pushed a commit
to CCLDArjun/ggml
that referenced
this issue
Dec 18, 2023
* ggml_graph_compute: deprecate using ggml_context, try resolve issue ggerganov#287 * rewrite: no longer consider backward compitability; plan and make_plan * minor: rename ctx as plan; const * remove ggml_graph_compute from tests/test-grad0.c, but current change breaks backward * add static ggml_graph_compute_sugar() * minor: update comments * reusable buffers * ggml : more consistent naming + metal fixes * ggml : fix docs * tests : disable grad / opt + minor naming changes * ggml : add ggml_graph_compute_with_ctx() - backwards compatible API - deduplicates a lot of copy-paste * ci : enable test-grad0 * examples : factor out plan allocation into a helper function * llama : factor out plan stuff into a helper function * ci : fix env * llama : fix duplicate symbols + refactor example benchmark * ggml : remove obsolete assert + refactor n_tasks section * ggml : fix indentation in switch * llama : avoid unnecessary bool * ggml : remove comments from source file and match order in header --------- Co-authored-by: Georgi Gerganov <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
We now have the following signature:
The provided context is used during the computation to potentially allocate a work buffer needed by some of the
ggmloperators. Not only the buffer is not always needed, but having to pass an entire context is a poor design choice.We need to avoid this by allowing the user to pass a work buffer (pointer and size) and an API that allows the user to query what is the work size needed to evaluate a specific compute graph. This way, the user will first make the query and then will be responsible to provide the work buffer externally if necessary
The text was updated successfully, but these errors were encountered: