Surreal Robotics Suite is a tookit and simulation benchmark powered by the MuJoCo physics engine for reproducible robotics research. The current release concentrates on reinforcement learning for robot manipulation. This library is designed to smoothly interoperate with the Surreal Distributed Reinforcement Learning Framework.
Reinforcement learning has been a powerful and generic tool in robotics. Reinforcement learning combined with deep neural networks, i.e. deep reinforcement learning (DRL), has achieved some exciting successes in a variety of robot control problems. However, the challenges of reproducibility and replicability in DRL and robotics have impaired research progress. Our goal is to provide an accessible set of benchmarking tasks that facilitates a fair and rigorus evaluation and improves our understanding of new methods.
This framework was originally developed since late 2017 by researchers in Stanford Vision and Learning Lab (SVL) as an internal tool for robot learning research. Today it is actively maintained and used for robotics research projects in SVL.
This release of Surreal Robotics Suite contains a set of benchmarking manipulation tasks and a modularized design of APIs for building new environments. We highlight these primary features below:
- standardized tasks: a set of single-arm and bimanual manipulation tasks of large diversity and varying complexity.
- procedural generation: modularized APIs for programmatically creating new scenes and new tasks as a combinations of robot models, arenas, and parameterized 3D objects;
- controller modes: a selection of controller types to command the robots, such as joint velocity control, inverse kinematics control, and 3D motion devices for teleoperation;
- multi-modal sensors: heterogeneous types of sensory signals, including low-level physical states, RGB cameras, depth maps, and proprioception;
- human demonstrations: utilities for collecting human demonstrations, replaying demonstration datasets, and leveraging demonstration data for learning.
Surreal Robotics Suite officially supports Mac OS X and Linux on Python 3.5 or 3.7. It can be run with an on-screen display for visualization or in a headless mode for model training, with or without a GPU.
The base installation requires the MuJoCo physics engine (with mujoco-py, refer to link for troubleshooting the installation and further instructions) and numpy. To avoid interfering with system packages, it is recommended to install it under a virtual environment by first running virtualenv -p python3 . && source bin/activate
.
First download MuJoCo 1.5.0 (Linux and Mac OS X) and place the mjpro150
folder and your license key mjkey.txt
in ~/.mujoco
. You can obtain a license key from here.
- For Linux, you will need to install some packages to build
mujoco-py
(sourced from here, with a couple missing packages added). If usingapt
, the required installation command is:Note that for older versions of Ubuntu (e.g., 14.04) there's no libglfw3 package, in which case you need to$ sudo apt install curl git libgl1-mesa-dev libgl1-mesa-glx libglew-dev \ libosmesa6-dev software-properties-common net-tools unzip vim \ virtualenv wget xpra xserver-xorg-dev libglfw3-dev patchelf
export LD_LIBRARY_PATH=$HOME/.mujoco/mjpro150/bin
before proceeding to the next step.
- After setting up mujoco, robosuite can be installed with
$ pip install robosuite
- Test your installation with
$ python -m robosuite.demo
- Clone the robosuite repository
$ git clone https://github.com/StanfordVL/robosuite.git
$ cd robosuite
-
Install the base requirements with
$ pip3 install -r requirements.txt
This will also install our library as an editable package, such that local changes will be reflected elsewhere without having to reinstall the package.
-
(Optional) We also provide add-on functionalities, such as OpenAI Gym interfaces, inverse kinematics controllers powered by PyBullet, and teleoperation with SpaceMouse devices (Mac OS X only). To enable these additional features, please install the extra dependencies by running
$ pip3 install -r requirements-extra.txt
-
Test your installation with
$ python robosuite/demo.py
The APIs we provide to interact with our environments are simple and similar to the ones used by OpenAI Gym. Below is a minimalistic example of how to interact with an environment.
import numpy as np
import robosuite as suite
# create environment instance
env = suite.make("SawyerLift", has_renderer=True)
# reset the environment
env.reset()
for i in range(1000):
action = np.random.randn(env.dof) # sample random action
obs, reward, done, info = env.step(action) # take action in the environment
env.render() # render on display
The step()
function takes an action
as input and returns a tuple of (obs, reward, done, info)
where obs
is an OrderedDict
containing observations [(name_string, np.array), ...]
, reward
is the immediate reward obtained per step, done
is a Boolean flag indicating if the episode has terminated and info
is a dictionary which contains additional metadata.
There are other parameters which can be configured for each environment. They provide functionalities such as headless rendering, getting pixel observations, changing camera settings, using reward shaping, and adding extra low-level observations. Please refer to this page and the environment classes for further details.
Sample scripts that showcase various features of the Surreal Robotics Suite are available at robosuite/scripts. The purpose of each script and usage instructions can be found at the beginning of each file.
A manipulation task
typically involves the participation of a robot
with gripper
s as its end-effectors, an arena
(workspace), and object
s that the robot interacts with. Our APIs in Models provide a toolkit of composing these modularized elements into a scene, which can be loaded in MuJoCo for simulation. To build your own environments, you are recommended to take a look at the environment classes which have used these APIs to define a set of standardized manipulation tasks. You can also find detailed documentations about creating a custom object and creating a custom environment.
We provide teleoperation utilities that allow users to control the robots with input devices, such as the keyboard and the SpaceMouse. Such functionality allows us to collect a dataset of human demonstrations for learning. We provide an example script to illustrate how to collect demonstrations. Our collect_human_demonstrations script takes the following arguments:
directory:
path to a folder for where to store the pickle file of collected demonstrationsenvironment:
name of the environment you would like to collect the demonstrations fordevice:
either "keyboard" or "spacemouse"
Our twin project RoboTurk has collected pilot datasets of more than a thousand demonstrations for two tasks in our Suite via crowdsourcing. You can find detailed information about the RoboTurk datasets and demonstration collection here.
We have included an example script that illustrates how demonstrations can be loaded and played back. Our playback_demonstrations_from_hdf5 script selects demonstration episodes at random from a demonstration pickle file and replays them. We have included some sample demonstrations for each task at models/assets/demonstrations
.
Several prior works have demonstrated the effectiveness of altering the start state distribution of training episodes for learning RL policies. We provide a generic utility for setting various types of learning curriculums which dictate how to sample from demonstration episodes when doing an environment reset. For more information see the DemoSamplerWrapper
class. We have provided an example of how to use this wrapper along with a demonstration pickle file in the demo_learning_curriculum script.
Please cite Surreal if you use this repository in your publications:
@inproceedings{corl2018surreal,
title={SURREAL: Open-Source Reinforcement Learning Framework and Robot Manipulation Benchmark},
author={Fan, Linxi and Zhu, Yuke and Zhu, Jiren and Liu, Zihua and Zeng, Orien and Gupta, Anchit and Creus-Costa, Joan and Savarese, Silvio and Fei-Fei, Li},
booktitle={Conference on Robot Learning},
year={2018}
}
Please also cite RoboTurk if you use the demonstration datasets:
@inproceedings{corl2018roboturk,
title={RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitation},
author={Mandlekar, Ajay and Zhu, Yuke and Garg, Animesh and Booher, Jonathan and Spero, Max and Tung, Albert and Gao, Julian and Emmons, John and Gupta, Anchit and Orbay, Emre and Savarese, Silvio and Fei-Fei, Li},
booktitle={Conference on Robot Learning},
year={2018}
}