gae formula bug

[kangnil]

annotated_deep_learning_paper_implementations/labml_nn/rl/ppo/gae.py

Line 36 in 999f203

\hat{A_t^{(\infty)}} &= r_t + \gamma r_{t+1} +\gamma^2 r_{t+1} + ... - V(s)

this should be gamma^2 r_{t+2} instead of gamma^2 r_{t+1}

annotated_deep_learning_paper_implementations/labml_nn/rl/ppo/gae.py

Line 45 in 999f203

We set $w_k = \lambda^{k-1}$, this gives clean calculation for

this should be wk = (1-lambda) * lambda ^ (k-1)

thanks for everything! this tutorial is gold!!!

[vpj]

Thanks fixed it here 20494ae