This is a clean and robust Pytorch implementation of PPO on Discrete action space. Here is the result:

All the experiments are trained with same hyperparameters. Other RL algorithms by Pytorch can be found here.

gymnasium==0.29.1
numpy==1.26.1
pytorch==2.1.0

python==3.11.5

python main.py

where the default enviroment is 'CartPole'.

python main.py --EnvIdex 0 --render True --Loadmodel True --ModelIdex 300000

which will render the 'CartPole'.

If you want to train on different enviroments

python main.py --EnvIdex 1

The --EnvIdex can be set to be 0 and 1, where

'--EnvIdex 0' for 'CartPole-v1'  
'--EnvIdex 1' for 'LunarLander-v2'   

Note: if you want train on LunarLander-v2, you need to install box2d-py first. You can install box2d-py via:

pip install gymnasium[box2d]

You can use the tensorboard to record anv visualize the training curve.

• Installation (please make sure PyTorch is installed already):

pip install tensorboard
pip install packaging

• Record (the training curves will be saved at '\runs'):

python main.py --write True

• Visualization:

tensorboard --logdir runs

For more details of Hyperparameter Setting, please check 'main.py'

Proximal Policy Optimization Algorithms
Emergence of Locomotion Behaviours in Rich Environments