you know, to walk the skeleton.
original DDPG/PPO code from https://github.com/ctmakro/gymnastics
more details at https://github.com/stanfordnmbl/osim-rl
École polytechnique fédérale de Lausanne (School Polytechnical Federal of Lausanne), the MIT of Switzerland. They seem to speak French.
- Python 3.5 (for training with TF on Windows)
- Python 2.7 (for running osim-rl on Windows)
- Anaconda/Miniconda (for the harmony of the two above)
- TF
- matplotlib
- pymsgbox (act as a stop button instead of using ctrl-c)
- Canton (No.1 DL library on earth)
- numpy, gym
- osim-rl (the competition)
- OpenCV3 (waterfall graph)
- Pyro4 (RPC)
- parse
- IPython (for convenience)
$ ipython -i ddpg2.py
then enter r(100)
to train the agent for 100 episodes.
For PPO implementation, just change the filename:
$ ipython -i ppo3.py
Current version requires farming (which is VERY IMPORTANT! Please read stanfordnmbl/osim-rl#58). Before starting ddpg2.py
, you should first start one or more farm, preferably with Python 2.7, by running python farm.py
on each SLAVE machine you own. Then create a farmlist.py
in the working directory (on the HOST machine) with the following content:
farmlist_base = [('127.0.0.1', 4),('192.168.1.33',8)]
# a farm of 4 cores is available on localhost, while a farm of 8 available on another machine.
# expand the list if you have more machines.
# this file will be consumed by the host to find the slaves.
Then start ipython -i ddpg2.py
. The farmer should be able to reach the farm on your local machine. You can now type r(100)
to train.
if you want to use the farming code for other algorithms/purposes, please see farming_demo.py
for an example.
Assume you want to run osim-rl on Windows w/py35, since TensorFlow support only Python 3.5 on Windows.
Edit: this is not encouraged. Start your env in Py27 and talk to it from Py35 is more sustainable.
-
Build OpenSim yourself (since @kidzik didn't do this for us)
-
Install VC++ 2015 build tools. Should not take long
-
CMD
-
Clone the opensim conda builder, then run the build command:
(chinese users) > set "HTTPS_PROXY=whatever_you_use" > git clone https://github.com/opensim-org/conda-opensim.git > cd conda-opensim > conda build opensim --python 3.5
Above will fail on Chinese Windows due to file encoding error. If so, don't run
conda build
again, as that will create a new build environment and pull the whole OpenSim from GitHub again.Instead, open
D:\Anaconda3\conda-bld\opensim_1499279773305\work\dependencies\BTK\Utilities\Open3DMotion\src\Open3DMotion\MotionFile\Formats\MDF\FileFormatMDF.cpp
and replace every occurence of the square symbol (superscript "2") with "^2".The file should now look like:
//... { "Marker Acceleration", "m/s^2", "%.3fm/s2", "%.3f", (float)0.05}, //...
-
Run the build script generated by the build process above again:
> cd D:\Anaconda3\conda-bld\opensim_1499279773305(actual-path-may-vary)\work > bld.bat
The compilation should run smoothly this time.
-
Since CMD was used to run
bld.bat
, you may notice that some commands, such as ancp
(copy) did not run successfully. It should produce errors like that:(d:\Anaconda3\conda-bld\opensim_1499279773305\_b_env) Nonecp d:\Anaconda3\conda-bld\opensim_1499279773305\_b_env\Library \simbody\bin\simbody-visualizer.exe d:\Anaconda3\conda-bld\opensim_1499279773305\_b_env\simbody-visualizer.exe 'cp' is not recognized as an internal or external command, operable program or batch file.`
you can do the copy manually, though.
-
also the
python setup.py install
might not run smoothly:(d:\Anaconda3\conda-bld\opensim_1499279773305\_b_env) Nonepython setup.py install Traceback (most recent call last): File "setup.py", line 8, in <module> execfile('opensim/version.py') NameError: name 'execfile' is not defined
-
cause
setup.py
is written in py2, so it seems that i built the py2(default) branch of OpenSim. -
solution
in
setup.py
, replaceexecfile(name)
withexec(open(name).read())
.
Then
cd
to the correct path (D:\Anaconda3\conda-bld\opensim_1499279773305\_b_env\Library\sdk\python
) and manually runpip install -e .
-
-
Now test the OpenSim installation:
-
try
import opensim
in python.upon
import opensim
python may fail to find simbody's libraries. -
solution
add
D:\Anaconda3\conda-bld\opensim_1499279773305\work\opensim_build\Release
to your PATH.
-
-
-
-
install osim-rl
-
CMD
(chinese users) > set HTTPS_PROXY=whatever_you_use > git clone https://github.com/stanfordnmbl/osim-rl.git > cd osim-rl > pip install -e . > cd tests > python test.manager.py
You should now see the skeleton swinging!
-
Update: this is not the goto choice since it introduce bias. your agent may not peform well during submission.
-
modify
D:\Anaconda3\conda-bld\opensim_1499279773305\work\OpenSim\Simulation\Manager\Manager.cpp
as follows:Manager::Manager(Model& model) : Manager(model, true) { SimTK::Integrator *vi = new SimTK::RungeKutta2Integrator(_model->getMultibodySystem()); vi->setAccuracy(3e-2); // reduce accuracy saves time _defaultInteg.reset(vi); _integ = *_defaultInteg; }
then build the whole thing again by running
bld.bat
mentioned above. -
parallelize the training environment (see code)
-
Windows users:
-
download precompiled Mesa3D for Windows at https://github.com/pal1000/mesa-dist-win/releases.
-
Self-extract that on your remote machine, in my case Windows Server 2008 R2.
-
Run the extracted
quickdeploy.cmd
. I installedopensim
for Py27 via a conda package built by kidzik (as per instructions on github/stanfordnmbl/osim-rl/README), into a conda virtual environment calledosrl
, therefore thesimbody-visualizer.exe
is located inc:\programdata\miniconda3\envs\osrl\
. Paste that path into the CMD window and let thequickdeploy.cmd
do the rest for you. -
Now you can start RunEnv(visualize=True) without getting any errors! Mesa3D just emulated OpenGL for you.
-
-
Linux users:
- Not much I could help, but the approach should be the same: if OpenGL is not supported, emulate it.