Traininig with rllib

[Maxwell2017]

Hi @Miffyli , I find rllib/configs/vizdoom_ppo.yaml in your repo, is this the config that you have verified which can use ppo algo （whthin RLlib） for training vizdoom? ：）

[Miffyli]

Yes, that is the config for PPO with rllib, but note that rllib was only used to run the continuous-control experiments in ViZDoom. For other experiments it used stable-baselines here, with all the arguments being default parameters from argparse here.

[Maxwell2017]

Get it ,PPO algorithm solves the problem of continuous action space，Therefore, it is not suitable to use ppo for the discrete motion space scenes such as basic and health gethering in Vizdoom.

[Miffyli]

More or less, yes (see the paper for results). Continuous spaces seem to be much harder to learn than discrete ones, so try to avoid them.

[Maxwell2017]

@Miffyli By the way, do you know the DQN training hyperparameters that can works well in other scenes of vizdoom (except basic and health gethering)? ：）

[Miffyli]

Sadly no, I have mainly used A2C or PPO for ViZDoom tasks lately. I think the default parameters used for Atari games should work reasonably well out-of-the-box, though :) .