This repository provides the source codes for our paper "Adaptive and Explainable Deployment of Navigation Skills via Hierarchical Deep Reinforcement Learning" in ICRA 2023.

For robotic vehicles to navigate robustly and safely in unseen environments, it is crucial to decide the most suitable navigation policy. However, most existing deep reinforcement learning based navigation policies are trained with a hand-engineered curriculum and reward function which are difficult to deploy in a wide range of real-world scenarios. In this paper, we propose a framework to learn a family of low-level navigation policies and a high-level policy for deploying them. The main idea is that, instead of learning a single navigation policy with a fixed reward function, we simultaneously learn a family of policies that exhibit different behaviors with a wide range of reward functions. We then train the high-level policy which adaptively deploys the most suitable navigation skill. We evaluate our approach in simulation and the real world and demonstrate that our method can learn diverse navigation skills and adaptively deploy them. We also illustrate that our proposed hierarchical learning framework presents explainability by providing semantics for the behavior of an autonomous agent.

The package has been tested on Ubuntu 18.04 / Python 3.6.

Please refer to nav-gym repo to install a navigation simulator. After finishing installation of nav-gym:

git clone https://github.com/leekwoon/hrl-nav.git

cd hrl-nav
pip install -e .

1. train low-level agent with:

python -m examples.train_low_level --base_logdir /tmp --seed 0

2. train high-level agent with:

# use correct path!
python -m examples.train_high_level --base_logdir /tmp --seed 0 --low_level_agent_params /tmp/...LOW_LEVEL_PATH.../itr_120.pkl

python -m examples.test --scenario atc --low_level_agent_params /tmp/...LOW_LEVEL_PATH.../itr_120.pkl --high_level_agent_params /tmp/...HIGH_LEVEL_PATH.../itr_80.pkl --spec atc

Kyowoon Lee*¹([email protected]), Seongun Kim*²([email protected]), Jaesik Choi²([email protected])

¹UNIST @ Department of Computer Science and Engineering, UNIST, Ulsan, Republic of Korea.

²KAIST @ Graduate School of Artificial Intelligence, KAIST, Daejeon, Republic of Korea.

* Equal contribution

@inproceedings{lee2023adaptive,
  title={Adaptive and Explainable Deployment of Navigation Skills via Hierarchical Deep Reinforcement Learning},
  author={Lee, Kyowoon and Kim, Seongun and Choi, Jaesik},
  booktitle={2023 IEEE International Conference on Robotics and Automation (ICRA)},
  pages={1673--1679},
  year={2023}
}

This repository is released under the MIT license. See LICENSE for additional details.

The codebase is extended from the rlkit codebase.