ggml_vec_dot_f16 - efficient way to implement this

[igsoft]

Hi,
not really an issue but more a curiosity,
instead of sumf += sum[0] + sum[1] + sum[2] + sum[3] + sum[4] + sum[5] + sum[6] + sum[7];

did you try for A64 something similar to this?
sumf = vaddvq_f32(vaddq_f32(vcvt_f32_f16(vget_low_f16(sum)), vcvt_high_f32_f16(sum)));

[ggerganov]

I believe you are looking at some outdated version.
Can you send a link to the exact line?

[igsoft]

You are right. I was just checking the homepage gpt-j where you wrote "Still, I'm curious to know if there is a more efficient way to implement this", but now I have seen that actually, you found the same solution. I don't know if the instruction vcvt_high_f32_f16 can give you an advantage in this line.