Skip to content

yiwenlu66/learning-qp

Repository files navigation

MPC-Inspired Reinforcement Learning for Verifiable Model-Free Control

Code for the paper: MPC-Inspired Reinforcement Learning for Verifiable Model-Free Control

Installation

git clone --recursive [email protected]:yiwenlu66/learning-qp.git
pip install -r requirements.txt

Note: the --recursive option is necessary to make the code work correctly.

Usage

python train_or_test env_name [--options]

The following scripts are also provided to reproduce the results in the paper:

  • experiments/tank/reproduce.sh for reproducing the first part of Table 1
  • experiments/cartpole/reproduce.sh for reproducing the second part of Table 1
  • experiments/tank/reproduce_disturbed.sh for reproducing Table 2

These scripts are run on GPU by default. After running each reproducing script, the following data will be saved:

  • Training logs in tensorboard format will be saved in runs
  • Test results, including the trial output for each experiment and a summary table, all in CSV format, will be saved in test_results

Code structure

  • rl_games: A customized version of the rl_games library for RL training
  • src/envs: GPU parallelized simulation environments, with interface similar to Isaac Gym
  • src/modules: PyTorch modules, including the proposed QP-based policy and the underlying differentiable QP solver
  • src/networks: Wrapper around the QP-based policy for interfacing with rl_games
  • src/utils: Utility functions (customized PyTorch operations, MPC baselines, etc.)
  • experiments: Sample scripts for running experiments

License

The project is released under the MIT license. See LICENSE for details.

Part of the project is modified from rl_games.

Citation

If you find this project useful in your research, please consider citing:

@InProceedings{lu2024mpc,
  title={MPC-Inspired Reinforcement Learning for Verifiable Model-Free Control},
  author={Lu, Yiwen and Li, Zishuo and Zhou, Yihan and Li, Na and Mo, Yilin},
  booktitle={Proceedings of the 6th Conference on Learning for Dynamics and Control},
  year={2024}
}

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published