Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ggml_status introduction #750

Merged
merged 7 commits into from
Mar 4, 2024

Conversation

Xarbirus
Copy link
Contributor

I would like to add the ability to explicitly specify the execution result for the ggml_backend_graph_plan_compute and ggml_backend_graph_compute functions.

This will allow us to determine whether the execution was aborted by abort_callback or by some other reasons (and allow us to use different reasons). I used the additional ggml_compute_result_t type because when executing ggml_*_compute multi-threaded, each thread may return its own error/warning, and in order not to lose them, I decided to concatenate exit_code's using OR.

Also I didn't remove GGML_EXIT_SUCCESS and GGML_EXIT_ABORTED because they can be used by other party libraries that use ggml.

@slaren
Copy link
Collaborator

slaren commented Feb 27, 2024

I think this is a good change. Some comments:

  • ggml_compute_exit_code does not need to be a bit field, in practice only one error can happen, and even if in the future there is some bizarre condition that may cause different errors in different threads, I think it is still better to report one error only to keep the API simpler
  • I would prefer a more generic name that can be reused in other ggml APIs in the future, such as ggml_error or ggml_status
  • Add a error_to_string function, otherwise every app will start implementing their own

@ggerganov
Copy link
Owner

+1 for ggml_status

ggml_compute_exit_code -> ggml_status
changed ggml_status from a bit-field type to simple codes
ggml_status to string cast
@Xarbirus
Copy link
Contributor Author

Thank you for your review, made changes:

  • renamed new type from ggml_compute_exit_code to ggml_status
  • added ggml_status_to_string (to cast a status to a string)
  • updated statuses, now negative values ​​mean errors and positive values ​​mean warnings

And I get an important question. The llama_decode function has its own error codes, for example -1 is used here and 1 is used here. Do I need to add all possible errors from llama.cpp to ggml_status or do I need to create a separate enum in llama.cpp that will expand and complement ggml_status later (after syncing ggml with llama.cpp)?

@slaren
Copy link
Collaborator

slaren commented Feb 28, 2024

My opinion is that llama.cpp needs its own llama_status with its own error codes, since llama.cpp has its own error conditions that are completely independent and separate of ggml.

@Xarbirus
Copy link
Contributor Author

@slaren , I completely agree with you!

src/ggml.c Outdated Show resolved Hide resolved
@Xarbirus Xarbirus changed the title Explicit ggml_compute exit codes ggml_status introduction Feb 28, 2024
@slaren slaren requested a review from ggerganov March 3, 2024 13:07
@ggerganov ggerganov merged commit 8695910 into ggerganov:master Mar 4, 2024
4 checks passed
ggerganov added a commit that referenced this pull request Mar 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants