[egs] Adding new non-splicing layers in some TDNN setups.

[danpovey]

I found that we can get the same improvement that @freewym was getting from adding non-splicing layers between each TDNN layer, by just adding two non-splicing layers towards the beginning of the network. There is even a small improvement on Switchboard.

I will probably merge this right away, as its an easy win. However there are a couple of outstanding issues.

• we should run this on some other setups as well and see if it improves them too.
• someone should test whether using just one of the two additional layers is sufficient.
• eventually we should try the same modification to other architectures such as TDNN+LSTM or TDNN+LSTM+attention.

Does anyone have time for this? @hhadian, maybe you could instruct one of the masters students how to run these experiments, they may learn something from the exercise.

[freewym]

For AMI SDM1, I found further increasing the num-epochs from 6 (default) to 8 is helpful:

System tdnn1g tdnn1g_nonsplicing tdnn1g_nonsplicing_epochs8
WER on dev 37.5 37.2 36.7
WER on eval 41.4 41.0 40.6
Final train prob -0.210036 -0.209284 -0.200288
Final valid prob -0.27294 -0.268485 -0.262626
Final train prob (xent) -2.46404 -2.41837 -2.35348
Final valid prob (xent) -2.69502 -2.66269 -2.61947

[danpovey]

Interesting. Try the same with just those initial nonsplicing layers. I do notice that these topologies might be underfitting a little bit.

…

On Fri, Oct 13, 2017 at 6:46 PM, Yiming Wang ***@***.***> wrote: For AMI SDM1, I found further increasing the num-epochs from 6 (default) to 8 is helpful: System tdnn1g tdnn1g_nonsplicing tdnn1g_nonsplicing_epochs8 WER on dev 37.5 37.2 36.7 WER on eval 41.4 41.0 40.6 Final train prob -0.210036 -0.209284 -0.200288 Final valid prob -0.27294 -0.268485 -0.262626 Final train prob (xent) -2.46404 -2.41837 -2.35348 Final valid prob (xent) -2.69502 -2.66269 -2.61947 — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#1937 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADJVu3Ro9oKq6K_SpkaE_RUIKJ81zRHoks5sr-hbgaJpZM4P5IKk> .