mnist result is not good

[jhyoocoder]

I got the performance of L2L better than that of Adam about quadratic problem.

1. python train.py --problem=quadratic --save_path=./quad

Run on GPU again
Epoch 10000
Log Mean Final Error: -1.08
Mean epoch time: 0.16 s
EVALUATION
Log Mean Final Error: -1.08
Mean epoch time: 0.05 s
Removing previously saved meta-optimizer
Saving meta-optimizer to ./quad

2. Evaluate with L2L; python evaluate.py --problem=quadratic --optimizer=L2L --path=./quad

Epoch 100
Log Mean Final Error: -0.70 # works better than Adam optimizer below
Mean epoch time: 0.17 s

3. Evaluate with Adam; python evaluate.py --problem=quadratic --optimizer=Adam --path=./quad

Epoch 100
Log Mean Final Error: -0.06
Mean epoch time: 0.09 s

However, the mnist problem seems not to reproduce the same result with that of the paper.
4. python train.py --problem=mnist --save_path=./mnist

Epoch 10000
Log Mean Final Error: -0.47
Mean epoch time: 0.76 s
EVALUATION
Log Mean Final Error: -0.42
Mean epoch time: 0.31 s

5. python evaluate.py --problem=mnist --optimizer=L2L --path=./mnist

Epoch 100
Log Mean Final Error: 0.18
Mean epoch time: 0.60 s

6. python evaluate.py --problem=mnist --optimizer=Adam --learning_rate=[0.1| 0.3 | 0.01 | 0.003 | 0.001]

0.1	0.3	0.01	0.003	0.001
Epoch 100 Log Mean Final Error: -0.31 Mean epoch time: 0.23 s	Epoch 100 Log Mean Final Error: 0.24 Mean epoch time: 0.23 s	Epoch 100 Log Mean Final Error: -0.31 Mean epoch time: 0.23 s	Epoch 100 Log Mean Final Error: 0.08 Mean epoch time: 0.24 s	Epoch 100 Log Mean Final Error: 0.29 Mean epoch time: 0.23 s

What did i do wrong? Could you help me? I resolved “TypeError: The two structures don't have the same nested structure.” with the solution mentioned in the following issue:
#22

Thank you in advance.

[jhyoocoder]

Resolved by myself. It was due to tensorflow version mismatch. I've worked on tf 1.14, but I succeeded after downgrading to tf1.5. Thanks!

[ko3n1g]

@jhyoocoder Can elaborate on your requirements.txt? I am facing the same issue - no reproducibility on mnist evaluate, am getting the same results like you initially

[ko3n1g]

solved with #34