Motivation : equation of REINFORCE and C.E is same
torch
C## ommand
python mnist_reinforce.py
: surprising performance jump of step :20, avg score : 0.27851563692092896 of step :40, avg score : 0.3832031190395355 of step :60, avg score : 0.4410156309604645 of step :80, avg score : 0.4769531190395355 of step :100, avg score : 0.537109375 of step :120, avg score : 0.5589843988418579 of step :140, avg score : 0.568359375 of step :160, avg score : 0.6117187738418579 of step :180, avg score : 0.635937511920929 of step :200, avg score : 0.6382812261581421 of step :220, avg score : 0.639453113079071 of step :240, avg score : 0.6402343511581421 of step :260, avg score : 0.640625 of step :280, avg score : 0.639453113079071 of step :300, avg score : 0.640625 of step :320, avg score : 0.640625 of step :340, avg score : 0.640625 of step :360, avg score : 0.6402343511581421 of step :380, avg score : 0.6402343511581421 of step :400, avg score : 0.640625 of step :420, avg score : 0.639453113079071 of step :440, avg score : 0.6402343511581421 of step :460, avg score : 0.6402343511581421 of step :480, avg score : 0.640625 of step :500, avg score : 0.640625 of step :520, avg score : 0.6402343511581421 of step :540, avg score : 0.6402343511581421 of step :560, avg score : 0.639453113079071 of step :580, avg score : 0.6390625238418579 of step :600, avg score : 0.6402343511581421 of step :620, avg score : 0.6429687738418579 of step :640, avg score : 0.646484375 of step :660, avg score : 0.6484375 of step :680, avg score : 0.6480468511581421 of step :700, avg score : 0.6480468511581421 of step :720, avg score : 0.6484375 of step :740, avg score : 0.6480468511581421 of step :760, avg score : 0.6484375 of step :780, avg score : 0.6484375 of step :800, avg score : 0.6480468511581421 of step :820, avg score : 0.6480468511581421 of step :840, avg score : 0.6480468511581421 of step :860, avg score : 0.6480468511581421 of step :880, avg score : 0.6480468511581421 of step :900, avg score : 0.6480468511581421 of step :920, avg score : 0.6480468511581421 of step :940, avg score : 0.647656261920929 of step :960, avg score : 0.6480468511581421 of step :980, avg score : 0.6484375 of step :1000, avg score : 0.6484375 of step :1020, avg score : 0.6484375 of step :1040, avg score : 0.647265613079071 of step :1060, avg score : 0.6484375 of step :1080, avg score : 0.6480468511581421 of step :1100, avg score : 0.6484375 of step :1120, avg score : 0.6484375 of step :1140, avg score : 0.6484375 of step :1160, avg score : 0.6484375 of step :1180, avg score : 0.6484375 of step :1200, avg score : 0.6480468511581421 of step :1220, avg score : 0.6484375 of step :1240, avg score : 0.6484375 of step :1260, avg score : 0.6480468511581421 of step :1280, avg score : 0.6480468511581421 of step :1300, avg score : 0.6480468511581421 of step :1320, avg score : 0.6484375 of step :1340, avg score : 0.6484375 of step :1360, avg score : 0.6484375 of step :1380, avg score : 0.6484375 of step :1400, avg score : 0.6488281488418579 of step :1420, avg score : 0.6480468511581421 of step :1440, avg score : 0.6480468511581421 of step :1460, avg score : 0.6484375 of step :1480, avg score : 0.6480468511581421 of step :1500, avg score : 0.6484375 of step :1520, avg score : 0.6480468511581421 of step :1540, avg score : 0.6480468511581421 of step :1560, avg score : 0.6484375 of step :1580, avg score : 0.6480468511581421 of step :1600, avg score : 0.657031238079071 of step :1620, avg score : 0.754687488079071 of step :1640, avg score : 0.819140613079071 of step :1660, avg score : 0.8324218988418579 of step :1680, avg score : 0.836718738079071 of step :1700, avg score : 0.836718738079071 of step :1720, avg score : 0.8382812738418579 of step :1740, avg score : 0.8382812738418579 of step :1760, avg score : 0.8421875238418579 of step :1780, avg score : 0.8441406488418579 of step :1800, avg score : 0.845703125 of step :1820, avg score : 0.84765625 of step :1840, avg score : 0.8492187261581421 of step :1860, avg score : 0.8492187261581421 of step :1880, avg score : 0.848828136920929 of step :1900, avg score : 0.849609375 of step :1920, avg score : 0.8492187261581421 of step :1940, avg score : 0.848828136920929 of step :1960, avg score : 0.848828136920929 of step :1980, avg score : 0.848828136920929 of step :2000, avg score : 0.850390613079071 of step :2020, avg score : 0.8480468988418579 of step :2040, avg score : 0.8480468988418579 of step :2060, avg score : 0.848437488079071 of step :2080, avg score : 0.853515625 of step :2100, avg score : 0.9066406488418579 of step :2120, avg score : 0.9566406011581421 of step :2140, avg score : 0.969531238079071 of step :2160, avg score : 0.978515625 of step :2180, avg score : 0.977734386920929 of step :2200, avg score : 0.983203113079071 of step :2220, avg score : 0.9859374761581421 of step :2240, avg score : 0.987109363079071 of step :2260, avg score : 0.9867187738418579 of step :2280, avg score : 0.9906250238418579 of step :2300, avg score : 0.996874988079071 of step :2320, avg score : 0.9937499761581421 of step :2340, avg score : 0.9945312738418579 of step :2360, avg score : 0.996874988079071 of step :2380, avg score : 0.9976562261581421 of step :2400, avg score : 0.997265636920929 of step :2420, avg score : 0.996874988079071 of step :2440, avg score : 0.998046875 of step :2460, avg score : 0.998046875 of step :2480, avg score : 0.998828113079071 of step :2500, avg score : 0.998046875 of step :2520, avg score : 0.9976562261581421 of step :2540, avg score : 0.998046875 of step :2560, avg score : 0.9984375238418579 of step :2580, avg score : 0.9984375238418579 of step :2600, avg score : 0.9976562261581421 of step :2620, avg score : 0.9976562261581421 of step :2640, avg score : 0.9964843988418579 of step :2660, avg score : 0.998046875 of step :2680, avg score : 0.998046875 of step :2700, avg score : 0.9996093511581421 of step :2720, avg score : 0.9996093511581421 of step :2740, avg score : 0.998828113079071 of step :2760, avg score : 0.9976562261581421 of step :2780, avg score : 0.999218761920929 of step :2800, avg score : 0.998828113079071 of step :2820, avg score : 0.9976562261581421 of step :2840, avg score : 0.999218761920929 of step :2860, avg score : 0.9996093511581421 of step :2880, avg score : 0.998828113079071 of step :2900, avg score : 0.9984375238418579 of step :2920, avg score : 0.997265636920929 of step :2940, avg score : 0.9976562261581421 of step :2960, avg score : 0.9984375238418579 of step :2980, avg score : 0.998046875 of step :3000, avg score : 0.9976562261581421 of step :3020, avg score : 0.9996093511581421 of step :3040, avg score : 0.9984375238418579 of step :3060, avg score : 0.999218761920929 of step :3080, avg score : 1.0 of step :3100, avg score : 0.9996093511581421 of step :3120, avg score : 0.998046875 of step :3140, avg score : 0.9996093511581421 of step :3160, avg score : 0.998828113079071 of step :3180, avg score : 0.9976562261581421 of step :3200, avg score : 0.9996093511581421 of step :3220, avg score : 0.998828113079071 of step :3240, avg score : 0.999218761920929 of step :3260, avg score : 0.998046875 of step :3280, avg score : 0.998828113079071 of step :3300, avg score : 0.999218761920929 of step :3320, avg score : 0.999218761920929 of step :3340, avg score : 0.9996093511581421 of step :3360, avg score : 0.9984375238418579 of step :3380, avg score : 0.9996093511581421 of step :3400, avg score : 0.999218761920929 of step :3420, avg score : 0.9976562261581421 of step :3440, avg score : 0.998828113079071 of step :3460, avg score : 0.9976562261581421 of step :3480, avg score : 0.9996093511581421 of step :3500, avg score : 0.999218761920929 of step :3520, avg score : 0.999218761920929 of step :3540, avg score : 1.0 of step :3560, avg score : 0.998046875 of step :3580, avg score : 0.999218761920929 of step :3600, avg score : 0.999218761920929 of step :3620, avg score : 0.996874988079071 of step :3640, avg score : 0.998828113079071 of step :3660, avg score : 0.9984375238418579 of step :3680, avg score : 0.998828113079071 of step :3700, avg score : 0.998828113079071 of step :3720, avg score : 0.998046875 of step :3740, avg score : 0.999218761920929 of step :3760, avg score : 0.9996093511581421 of step :3780, avg score : 0.998046875 of step :3800, avg score : 0.998046875 of step :3820, avg score : 0.9996093511581421 of step :3840, avg score : 0.9984375238418579 of step :3860, avg score : 0.9984375238418579 of step :3880, avg score : 0.999218761920929 of step :3900, avg score : 0.999218761920929 of step :3920, avg score : 0.9996093511581421 of step :3940, avg score : 0.999218761920929 of step :3960, avg score : 0.9984375238418579 of step :3980, avg score : 0.998046875 of step :4000, avg score : 0.9996093511581421 of step :4020, avg score : 0.999218761920929 of step :4040, avg score : 0.999218761920929 of step :4060, avg score : 0.9996093511581421 of step :4080, avg score : 0.9996093511581421 of step :4100, avg score : 0.9996093511581421 of step :4120, avg score : 0.998828113079071 of step :4140, avg score : 0.999218761920929 of step :4160, avg score : 0.998046875