Skip to content

Commit

Permalink
Correctly emit log probabilties for ctc beam search
Browse files Browse the repository at this point in the history 
This is not correctly documented in tensorflow.
See tensorflow/tensorflow#6929
  • Loading branch information
@louiskirsch
louiskirsch committed Apr 16, 2017
1 parent 9a43386 commit 36f40f9
Showing 1 changed file with 3 additions and 1 deletion.
4 changes: 3 additions & 1 deletion speech_model.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
# ==============================================================================

import tensorflow as tf
import math
import abc

from tensorflow.contrib.layers import xavier_initializer
Expand Down Expand Up @@ -78,7 +79,8 @@ def __init__(self, input_loader, input_size, num_classes, learning_rate, learnin
# Decoding
with tf.name_scope('decoding'):
if language_model:
self.decoded, self.log_probabilities = tf.nn.ctc_beam_search_decoder(self.logits,
self.softmaxed = tf.log(tf.nn.softmax(self.logits, name='softmax') + 1e-8) / math.log(10)
self.decoded, self.log_probabilities = tf.nn.ctc_beam_search_decoder(self.softmaxed,
self.sequence_lengths // 2,
kenlm_directory_path=language_model,
beam_width=100,
Expand Down

1 comment on commit 36f40f9

@MarkDaoust
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi,

I was just looking at the linked issue and it's clear from the code that un-normalized log-probabilities (logits) are fine. So this change was probably unnecessary.

But that /log(10) is a little funny, you can add an offset to logits without changing the underlying probabilities, but multiplying by a factor like that will change them.

Please sign in to comment.