Skip to content

Commit

Permalink
ggml : prevent builds with -ffinite-math-only (llama/7726)
Browse files Browse the repository at this point in the history
This enforces a check that -fno-finite-math-only was set and that the operating
compiling mode is not in finite maths mode. This is because during rewriting of
silu and softmax for cpu #7154 there emerged an issue where the result that was
observed when >1 slot was nondeterministic as found by @JohannesGaessler.

@LostRuins narrowed the problem down to -ffinite-math-only which was theorised
to be due to SiLU, instead of flushing small values to 0, returns NaN or some
other garbage. @jart proposed a fix that @ggerganov then implemented in this fix

ref ggerganov/llama.cpp#7154 (comment)
  • Loading branch information
ggerganov committed Jun 15, 2024
1 parent 8d8d6b1 commit f3c91f6
Showing 1 changed file with 5 additions and 0 deletions.
5 changes: 5 additions & 0 deletions src/ggml.c
Original file line number Diff line number Diff line change
Expand Up @@ -2272,6 +2272,11 @@ inline static float ggml_silu_f32(float x) {
return x/(1.0f + expf(-x));
}

#if __FINITE_MATH_ONLY__
#error "some routines in ggml.c require non-finite math arithmetics -- pass -fno-finite-math-only to the compiler to fix"
#error "ref: https://github.com/ggerganov/llama.cpp/pull/7154#issuecomment-2143844461"
#endif

#if defined(__ARM_NEON) && defined(__aarch64__)

// adapted from arm limited optimized routine
Expand Down

0 comments on commit f3c91f6

Please sign in to comment.