Directly converted from bfloat16 weights are 20x slower than converted from float32 ones.

[OlegJakushkin]

Sounds like magic - yet:

Save model to bfloat16 in python -> run convert-h5-to-ggml passing 1 as target -> run ggml model and get 983.91 ms per token
Get q8_0 quantised model from that f16 artifact -> 21ms per token
vs
Save model to float32 -> run convert-h5-to-ggml passing 1 as target -> run ggml model 41.68 ms per token

tested on mpt model. So it is solvable by saving to float32 in torch, but why?