Harley Seal AVX-512 implementations #138
Conversation
This commit adds the optimized Harley Seal kernel from the `WojciechMula/sse-popcount` library to the benchmarking suite to investigate optimization opportunities on Intel Sapphire Rapids and AMD Genoa chips.
| @@ -223,6 +221,182 @@ void vdot_f64c_blas(simsimd_f64_t const* a, simsimd_f64_t const* b, simsimd_size | |||
|
|
|||
| #endif | |||
|
|
|||
| namespace AVX512_harley_seal { | |||
|
|
|||
| uint8_t lookup8bit[256] = { | |||
There was a problem hiding this comment.
Does it help to add const/constexpr? I wonder if it would encourage the table to be cached. It might also help to run a loop over it to pre-load it into cache too (although I figure prefetching would most likely get the whole table in the first access).
In my own experiments in the past, I did find the built in instructions to be faster vs LUTs, however.
There was a problem hiding this comment.
Assuming the size of the inputs - the tail will never be evaluated separately. I've just copied that part of the code for completeness.
| uint64_t lower_qword(const __m128i v) { return _mm_cvtsi128_si64(v); } | ||
|
|
||
| uint64_t higher_qword(const __m128i v) { return lower_qword(_mm_srli_si128(v, 8)); } | ||
|
|
||
| uint64_t simd_sum_epu64(const __m128i v) { return lower_qword(v) + higher_qword(v); } | ||
|
|
||
| uint64_t simd_sum_epu64(const __m256i v) { |
There was a problem hiding this comment.
I think modern compilers might do this without asking in some cases, but using inline might encourage it (and could help with these small functions).
There was a problem hiding this comment.
The changes I've suggested so far are just low hanging fruit though. Have you used profiling tools to find which lines of code each approach is spending the most time in?
There was a problem hiding this comment.
Most time is spent in the main loop computing CSAs. Sadly, I can't access hardware performance counters on those machines.
Binary representations are becoming increasingly popular in Machine Learning and I'd love to explore the opportunity for faster Hamming and Jaccard distance calculations. I've looked into several benchmarks, most importantly the
WojciechMula/sse-popcountlibrary, that compares several optimizations for population-counts -the most expensive part of the Hamming/Jaccard kernel.Extensive benchmarks and the design itself suggest that AVX-512 Harley Seal variant should be the fastest on long inputs beyond 1 KB. Here is a sample of the most recent results obtained on an i3 Cannonlake Intel CPU:
I've tried copying the best solution into SimSIMD benchmarking suite and sadly didn't achieve similar improvements on more recent CPUs. On Intel Sapphire Rapids CPUs:
On AMD Genoa:
_mm_popcnt_u64._mm512_popcnt_epi64.icehsis an adaptation of the Harley Seal transform that "zip"-s two input streams withxor.To reproduce the results:
Please let me know if there is a better way to accelerate this kernel 🤗