Only check the actual used length of the hash

[seism0saurus]

This fixes the error in #8707 but needs a professional review, since I am no C programmer.

The signature check fails, if the hash->len() is only 32 bytes instead of the assumed 64 bytes.
Since the variable HashLengthMax has a max in its name, I assume that it is allowed to have shorter hash lengths.
I assume that memcmp did an buffer over read here and read stuff after the 32 bytes of hash. But that's my narrow understanding of the internals of C as a non-C developer.

[mcspr]

Thanks!

[mcspr]

I assume that memcmp did an buffer over read here and read stuff after the 32 bytes of hash.

Right. No function called to fill it (memset, std::fill), no zero-init (= {0};, = {};) or explicit member init (u8 buf[] {1,2,3}), so we end up with some random stack contents. Also, accidentally read 32bytes past hash object real buffer contents.