-
Notifications
You must be signed in to change notification settings - Fork 130
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
about the weight norm #1
Comments
Because we directly change the weights in the forward propagation. We find the performance is similar with calculating the gradients of the normalization, and it looks much clearer. You may also try a more correct way as in https://github.com/happynear/NormFace/blob/master/prototxt/combine_softmax_contrastive.prototxt . |
thanks. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
hi @happynear,
i read the code of
innerproduct
, find that, you only norm theweights
in theforward
pass. so why it does not need correspondingbp
?thanks.
The text was updated successfully, but these errors were encountered: