Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ggml : change ggml_graph_compute() API to not require context #1999

Merged
merged 20 commits into from
Jul 7, 2023
Merged

ggml : change ggml_graph_compute() API to not require context #1999

Changes from 1 commit
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
a1e7c69
ggml_graph_compute: deprecate using ggml_context, try resolve issue #287
mqy Jun 26, 2023
b11ac01
rewrite: no longer consider backward compitability; plan and make_plan
mqy Jul 3, 2023
a37de23
minor: rename ctx as plan; const
mqy Jul 3, 2023
db81f33
remove ggml_graph_compute from tests/test-grad0.c, but current change…
mqy Jul 3, 2023
2b502c3
add static ggml_graph_compute_sugar()
mqy Jul 3, 2023
cb1dec0
minor: update comments
mqy Jul 3, 2023
b1331d7
reusable buffers
mqy Jul 4, 2023
53cfb4b
ggml : more consistent naming + metal fixes
ggerganov Jul 6, 2023
4646cc2
ggml : fix docs
ggerganov Jul 6, 2023
8e1f0b6
tests : disable grad / opt + minor naming changes
ggerganov Jul 6, 2023
2392f7a
ggml : add ggml_graph_compute_with_ctx()
ggerganov Jul 6, 2023
1b9994f
ci : enable test-grad0
ggerganov Jul 6, 2023
a67404e
examples : factor out plan allocation into a helper function
ggerganov Jul 6, 2023
2d3a525
llama : factor out plan stuff into a helper function
ggerganov Jul 6, 2023
8fdf86d
ci : fix env
ggerganov Jul 6, 2023
9c9bdaf
llama : fix duplicate symbols + refactor example benchmark
ggerganov Jul 6, 2023
8dc7f10
ggml : remove obsolete assert + refactor n_tasks section
ggerganov Jul 6, 2023
551ed08
ggml : fix indentation in switch
ggerganov Jul 6, 2023
f789f2c
llama : avoid unnecessary bool
ggerganov Jul 6, 2023
c15833c
ggml : remove comments from source file and match order in header
ggerganov Jul 7, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
llama : avoid unnecessary bool
  • Loading branch information
@ggerganov
ggerganov committed Jul 6, 2023
commit f789f2cef2e40ef9577b29688a55be350849ea99
9 changes: 3 additions & 6 deletions llama.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1613,14 +1613,11 @@ static bool llama_eval_internal(
// run the computation
ggml_build_forward_expand(&gf, cur);

bool call_ggml_graph_compute = true;

#ifdef GGML_USE_METAL
if (lctx.ctx_metal && N == 1) {
ggml_metal_set_n_cb (lctx.ctx_metal, n_threads);
ggml_metal_graph_compute(lctx.ctx_metal, &gf);
ggml_metal_get_tensor (lctx.ctx_metal, cur);
call_ggml_graph_compute = false;
} else {
// IMPORTANT:
// Since we don't have efficient Matrix x Matrix Metal multiplication yet, we fallback to vanilla
Expand All @@ -1637,12 +1634,12 @@ static bool llama_eval_internal(
ggml_metal_get_tensor(lctx.ctx_metal, kv_self.k);
ggml_metal_get_tensor(lctx.ctx_metal, kv_self.v);
}
}
#endif

if (call_ggml_graph_compute) {
ggml_graph_compute_helper(lctx.work_buffer, &gf, n_threads);
}
#else
ggml_graph_compute_helper(lctx.work_buffer, &gf, n_threads);
#endif

if (cgraph_fname) {
ggml_graph_export(&gf, cgraph_fname);
Expand Down
Loading