Build failiure on armv7 (docker)

[walking-octopus]

Hi! Thank you for your work on ggml and whisper.cpp, these two amazing projects really did wonders for the performance. I tried compiling the examples to see if they can be run on an Ubuntu Touch phone, which is essentially an Ubuntu 16.04 LTS Core on arm. Sadly, the build failed due to not finding immintrin.h, which as far as I understood is present only on x84 systems. As you can tell, I am a novice, so I'd appreciate any help regarding this. Once again, thanks for all your work! :)

Logs

root@d1fa92e4b731:~/ggml/build# make -j4 gpt-2 gpt-j
[ 50%] Built target ggml_utils
[ 50%] Building C object src/CMakeFiles/ggml.dir/ggml.c.o
/root/ggml/src/ggml.c:153:23: fatal error: immintrin.h: No such file or directory
compilation terminated.
src/CMakeFiles/ggml.dir/build.make:62: recipe for target 'src/CMakeFiles/ggml.dir/ggml.c.o' failed
make[3]: *** [src/CMakeFiles/ggml.dir/ggml.c.o] Error 1
CMakeFiles/Makefile2:85: recipe for target 'src/CMakeFiles/ggml.dir/all' failed
make[2]: *** [src/CMakeFiles/ggml.dir/all] Error 2
CMakeFiles/Makefile2:622: recipe for target 'examples/gpt-2/CMakeFiles/gpt-2.dir/rule' failed
make[1]: *** [examples/gpt-2/CMakeFiles/gpt-2.dir/rule] Error 2
Makefile:329: recipe for target 'gpt-2' failed
make: *** [gpt-2] Error 2
root@d1fa92e4b731:~/ggml/build# uname -a
Linux d1fa92e4b731 5.19.0-26-generic #27-Ubuntu SMP PREEMPT_DYNAMIC Wed Nov 23 20:44:15 UTC 2022 armv7l armv7l armv7l GNU/Linux

[ggerganov]

Try with the following commands:

git clone https://github.com/ggerganov/ggml
cd ggml

# build gpt-2
gcc -std=c11 -O3 -I include/ggml/ -pthread -mfpu=neon-fp-armv8 -mfp16-format=ieee -mno-unaligned-access -funsafe-math-optimizations -c src/ggml.c -o ggml.o
g++ -O3 -pthread -I include/ -I examples/ examples/gpt-2/main.cpp examples/utils.cpp ggml.o -o gpt-2

# download model
./examples/gpt-2/download-ggml-model.sh 117M

# run with default parameters
./gpt-2

[walking-octopus]

Sadly, it still failed. Also, would the resulting binary work on armv7? Are those older phones using it, like the Nexus 5, even powerful enough for these models?

Logs

root@aa25a7be191b:/home/ggml# g++ -O3 -pthread -I include/ -I examples/ examples/gpt-2/main.cpp examples/utils.cpp ggml.o -o gpt-2
g++: error: ggml.o: No such file or directory
root@aa25a7be191b:/home/ggml# gcc -std=c11 -O3 -I include/ggml/ -pthread -mfpu=neon-fp-armv8 -mfp16-format=ieee -mno-unaligned-access -funsafe-math-optimizations -c src/ggml.c -o ggml.o
In file included from src/ggml.c:1:0:
include/ggml/ggml.h:194:1: error: function parameters cannot have __fp16 type
 float       ggml_fp16_to_fp32(ggml_fp16_t x);
 ^
include/ggml/ggml.h:195:1: error: functions cannot return __fp16 type
 ggml_fp16_t ggml_fp32_to_fp16(float x);
 ^
src/ggml.c:271:1: error: function parameters cannot have __fp16 type
 float ggml_fp16_to_fp32(ggml_fp16_t x) {
 ^
src/ggml.c:275:1: error: functions cannot return __fp16 type
 ggml_fp16_t ggml_fp32_to_fp16(float x) {
 ^
src/ggml.c: In function 'ggml_time_ms':
src/ggml.c:304:5: warning: implicit declaration of function 'clock_gettime' [-Wimplicit-function-declaration]
     clock_gettime(CLOCK_MONOTONIC, &ts);
     ^
src/ggml.c:304:19: error: 'CLOCK_MONOTONIC' undeclared (first use in this function)
     clock_gettime(CLOCK_MONOTONIC, &ts);
                   ^
src/ggml.c:304:19: note: each undeclared identifier is reported only once for each function it appears in
src/ggml.c: In function 'ggml_time_us':
src/ggml.c:310:19: error: 'CLOCK_MONOTONIC' undeclared (first use in this function)
     clock_gettime(CLOCK_MONOTONIC, &ts);

[ggerganov]

These were the commands that I used to make it run on Raspberry Pi 4 and it worked. I was hoping it would work in your case too, but likely some flag is missing.

You can build this code for pretty much every device you want.
It's just a matter of using the correct gcc flags.