-
Notifications
You must be signed in to change notification settings - Fork 18
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
No weight update due to sign function #6
Comments
Sorry, for the late reply. Yes, I didn't get time to fix that one but I guess your solution will work, will try and publish the results. |
@jonkoi You are right, the implementation is not correct. The first two ABC layers behave only as constant feature extractors and all the learning happens in the last two float fully connected layers. |
Even with the weight update, it seems to not be learning on any dataset beside MNIST. Did you have success @jonkoi? |
Hi,
I don't think weights get updated because of the tf.sign function cutting off backprop gradients:
binarized_filters = tf.sign(tiled_filters + expanded_stddev, name="binarized_filters")
i believe this line is needed:
with tf.get_default_graph().gradient_override_map({"Sign": "Identity"}):
The text was updated successfully, but these errors were encountered: