-
Notifications
You must be signed in to change notification settings - Fork 67
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
BERT-flow是什么原理,还是不懂。 #20
Comments
刚看了下,浅答一下,有什么问题的话,望大佬轻喷paper :https://arxiv.org/pdf/2011.05864.pdf要看懂这个论文,主要看下图1,公式4。说的很清楚了,将 BERT sentence embeddings 映射到 标准高斯分布空间。 无监督的话主要看下model_fn_builder |
大佬,有监督 和 无监督 的 训练目标 各是什么啊? |
目标是一致的,无监督和有监督在这里的区别就是是否把监督学习的 loss 算进去,model_fn_builder。 你具体要问的东西可能是无监督具体在学什么,应该在 objective_tower 函数。 with arg_scope(ops, init=init):
encoder = glow_ops.encoder_decoder
self.z, encoder_objective, self.eps, _, _ = encoder(
"flow", x, self.hparams, eps=None, reverse=False)
objective += encoder_objective
self.z_top_shape = get_shape_list(self.z)
prior_dist = self.top_prior()
prior_objective = tf.reduce_sum(
prior_dist.log_prob(self.z), axis=[1, 2, 3])
#self.z_sample = prior_dist.sample()
objective += prior_objective
# bits per pixel
_, h, w, c = get_shape_list(x)
objective = -objective / (np.log(2) * h * w * c) 主要看下这几行 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
多谢啊
The text was updated successfully, but these errors were encountered: