Is kNormLimitColumnwise performing the correct reduction?

[lzamparo]

I looked at your code for max-norm regularization after reading section 2.2 of your thesis (and the papers you cited), and was puzzled by how the CUDA kernel performs the computation.

As I understand it, the max-norm of each column is limited to some value B by ensuring that the column with the maximal sum of squares is re-scaled so that its norm does not exceed B. But your kernel does a sum reduction (c.f line 1068) on the sum of squares of the columns rather than a max reduction. This seems like it will lead to more stringent / severe regularization than was intended by the authors of the max-norm paper.

Am I misunderstanding the code? Or are you using the wrong reduction (sum vs max) here?

Thanks for clearing this up :)