Skip to content

Scitator/run-skeleton-run-in-3d

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

NeurIPS 2019: Learn to Move – Walk Around
2nd place solution – powered by Catalyst.RL

alt text

Telegram Gitter Slack

How2run

System requirements

You need to install Anaconda and Redis:

sudo apt install redis-server

Python requirements

(Taken from the official repo).

Anaconda is required to run our simulations. Anaconda will create a virtual environment with all the necessary libraries, to avoid conflicts with libraries in your operating system. You can get anaconda from here https://docs.anaconda.com/anaconda/install/. In the following instructions we assume that Anaconda is successfully installed.

For the challenge we prepared OpenSim binaries as a conda environment to make the installation straightforward

We support Windows, Linux, and Mac OSX (all in 64-bit). To install our simulator, you first need to create a conda environment with the OpenSim package.

On Windows, open a command prompt and type:

conda create -n opensim-rl -c kidzik opensim python=3.6.1
activate opensim-rl

On Linux/OSX, run:

conda create -n opensim-rl -c kidzik opensim python=3.6
source activate opensim-rl
conda install python=3.6.1 -c conda-forge

These commands will create a virtual environment on your computer with the necessary simulation libraries installed. Next, you need to install our python reinforcement learning environment. Type (on all platforms):

conda install -c conda-forge lapack git -y
pip install osim-rl -y
conda remove nb_conda_kernels -y
conda install -c conda-forge nb_conda_kernels -y
conda install notebook jupyter nb_conda -y
conda remove nbpresent -y
pip install -r ./requirements.txt

Run ensemble training

alt text

Catalyst.RL achitecture. Samplers interact with the environment and gather training data. Trainers retrieve collected data and update parameters of value function and policy approximators. All communication is conducted through a database.

export LOGDIR=/path/to/logdir
export PORT=14001
bash bin/prepare_configs.sh

redis-server --port $PORT

CUDA_VISIBLE_DEVICES="0" \
PYTHONPATH="." \
EXP_CONFIG="./configs/_exp_common.yml ./configs/env_l2m.yml ./configs/_dpg_common.yml ./configs/td3.yml" \
DB_SPEC="null" \
OMP_NUM_THREADS=1 MKL_NUM_THREADS=1 catalyst-rl-run \
    --db/prefix=td3-quan-01-04:str \
    --environment/history_len=1:int \
    --environment/frame_skip=4:int

CUDA_VISIBLE_DEVICES="1" \
PYTHONPATH="." \
EXP_CONFIG="./configs/_exp_common.yml ./configs/env_l2m.yml ./configs/_dpg_common.yml ./configs/td3.yml" \
DB_SPEC="null" \
OMP_NUM_THREADS=1 MKL_NUM_THREADS=1 catalyst-rl-run \
    --db/prefix=td3-quan-04-04:str \
    --environment/history_len=4:int \
    --environment/frame_skip=4:int

CUDA_VISIBLE_DEVICES="2" \
PYTHONPATH="." \
EXP_CONFIG="./configs/_exp_common.yml ./configs/env_l2m.yml ./configs/_dpg_common.yml ./configs/td3.yml" \
DB_SPEC="null" \
OMP_NUM_THREADS=1 MKL_NUM_THREADS=1 catalyst-rl-run \
    --db/prefix=td3-quan-08-04:str \
    --environment/history_len=8:int \
    --environment/frame_skip=4:int

CUDA_VISIBLE_DEVICES="3" \
PYTHONPATH="." \
EXP_CONFIG="./configs/_exp_common.yml ./configs/env_l2m.yml ./configs/_dpg_common.yml ./configs/td3.yml" \
DB_SPEC="null" \
OMP_NUM_THREADS=1 MKL_NUM_THREADS=1 catalyst-rl-run \
    --db/prefix=td3-quan-12-04:str \
    --environment/history_len=12:int \
    --environment/frame_skip=4:int

Check results in Tensorboard

alt text

Average raw reward on validation seeds versus training time (in hours).
Different graphs correspond to different history lengths used for training.

Additional links [WIP]

  1. Catalyst.RL
  2. Analysis of the solution - slides (in English)
  3. Analysis of the solution - video (in Russian)
  4. Cool video with agent run
  5. NeurIPS 2019: Learn to Move - Walk Around – starter kit

Citation

Please cite the following paper if you feel this repository useful.

@article{run_skeleton_in3d,
  title={Sample Efficient Ensemble Learning with Catalyst.RL},
  author = {Kolesnikov, Sergey and Khrulkov, Valentin},
  journal={arXiv preprint arXiv:[WIP]},
  year={2019}
}

About

NeurIPS 2019: Learn to Move - Walk Around, 2nd place solution

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published