Fp16 support for layernorm

[eric-haibin-lin]

Currently, given fp16 inputs, nd.LayerNorm/sym.LayerNorm perform reduction in fp16, which losses precision. The reduction should be done in fp32 instead. @sxjscience

[mxnet-label-bot]

Hey, this is the MXNet Label Bot.
Thank you for submitting the issue! I will try and suggest some labels so that the appropriate MXNet community members can help resolve it.
Here are my recommended labels: Feature

[sxjscience]

Current I propose to solve the issue following these two steps:

• Add a new 'reduer_dtype' flag to the LayerNorm operator. When the flag is set, the reduction will be performed in the specified dtype. Always use the dtype-safe reductions in LayerNorm (Safe LayerNorm #14699)
• Accelerate the implementation of LayerNorm by taking advantage of the techniques in https://github.com/NVIDIA/apex/blob/master/csrc/layer_norm_cuda_kernel.cu, e.g., welford online sum (https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance)

[haojin2]

Once #14616 is merged then we can simply switch the LayerNorm's reduction to the safe version to achieve the 1st step @sxjscience proposed. Then we can possibly explore the implementation of the 2nd step later. @eric-haibin-lin What do you think?

[sxjscience]

@haojin2 Yes, should first try to directly change the reduce to the safe version.

[leezu]

This can probably be closed as #15002 is merged