Cannot replicate English to Vietnamese - on python 2 and 3

[mohammedayub44]

Hi,

Using the below configuration:
Python 2.7.15
Tensorflow: 1.10.0

Training Query:
python -m nmt.nmt --src=en --tgt=vi --hparams_path=/home/ubuntu/nmt/standard_hparams/iwslt15.json --vocab_prefix=/home/ubuntu/en_vi_dataset/vocab --train_prefix=/home/ubuntu/en_vi_dataset/train --dev_prefix=/home/ubuntu/en_vi_dataset/tst2012 --test_prefix=/home/ubuntu/en_vi_dataset/tst2013 --out_dir=/home/ubuntu/en_vi_dataset/models/attention_model27

Best Bleu score comes out to be: 1.5698. Not sure whats going wrong, hoping its not encoding as I used that files directly from the download.

Below file contains the Params and the Log for Python 2 run
run1_py_2.7.15.zip

Python 3 Run:
Python 3.6.5
Tensorflow: 1.10.0

Training Query:
python -m nmt.nmt --src=en --tgt=vi --hparams_path=/home/ubuntu/nmt/standard_hparams/iwslt15.json --vocab_prefix=/home/ubuntu/en_vi_dataset/vocab --train_prefix=/home/ubuntu/en_vi_dataset/train --dev_prefix=/home/ubuntu/en_vi_dataset/tst2012 --test_prefix=/home/ubuntu/en_vi_dataset/tst2013 --out_dir=/home/ubuntu/en_vi_dataset/models/attention_model

Best Bleu score comes out to be: 2.18338

Below file contains the Params and the Log for Python 3 run:
run2_py_3.6.5.zip

Appreciate any help.
Lopping some folks who have answered similar questions before.
@bastings @oahziur

Thanks !

Mohammed Ayub

[bastings]

Hi there,

I checked your python 3 logfile and hparams and the parameters that were used for training are quite a bit different from the original https://github.com/tensorflow/nmt/blob/master/nmt/standard_hparams/iwslt15.json

Are you specifying the configuration file correctly? It looks like your run did not use attention and only used 32 units.

[mohammedayub44]

@bastings Thanks.
My bad, silly mistake, the path to the hparams file --hparams_path=/home/ubuntu/nmt/standard_hparams/iwslt15.json was incorrect, it should have been --hparams_path=/home/ubuntu/nmt/nmt/standard_hparams/iwslt15.json
Strangely, the train command did not throw an error but just used a default configuration. (not sure if this is a bug I guess its logged output some where but I did not see it).

I have made the changes and running it now. Will update you in sometime.

-Mohammed Ayub

[mohammedayub44]

@bastings Thanks for you help.
Ran perfectly fine. Bleu score a little off on the test side, but that might generalize with more runs I'm guessing.

Mohammed Ayub