sudo apt install -y xvfb
conda create -n groot python=3.9.16 -y
conda activate groot
conda install --channel=conda-forge openjdk=8 gym=0.19 -y
pip install git+https://github.com/karpathy/minGPT.git
git clone [email protected]:CraftJarvis/GROOT.git
pip install -e .Then, download simulator from [please contact us for the compiled simulator] and extract it.
tar -zxvf mcp.tar.gz -C jarvis/stark_tech/Then, run the simulator.
cd jarvis/stark_tech
xvfb-run -a python env_interface.pyIf you see the following logs, then the simulator is successfully installed.
[Close-ended] Slow reset with world seed: 19961103
Dict('buttons': MultiDiscrete([8641]), 'camera': MultiDiscrete([121]))
dict_keys(['pov', 'inventory', 'equipped_items', 'pickup', 'break_item', 'craft_item', 'mine_block', 'kill_entity', 'player_pos', 'is_gui_open'])
{'x': 541.5, 'y': 15.0, 'z': 358.5, 'pitch': 0.0, 'yaw': 0.0}
frame: 0, fps: 13.20, avg_fps: 13.20
frame: 50, fps: 26.53, avg_fps: 27.48
Simulator is being closed.
Here we provide some checkpoints for evaluation.
- GROOT_EFF_1x [1x weights A] [1x weights B]
- VPT_NATIVE [2x model] [early-game-finetuned]
Check the policy configs (jarvis/arm/configs) and assign the right checkpoint path.
# evaluate vanilla VPT model
export JARVISBASE_PRETRAINED="/path/to/ckptdir"
xvfb-run -a python run_agent.py --test 'run_test_vpt_single'
# evaluate GROOT model
xvfb-run -a python run_agent.py --test 'run_test_groot_single'
# evaluate GROOT model with the logit-subtraction trick
xvfb-run -a python run_agent.py --test 'run_test_groot_subtract'@article{cai2023groot,
title={Groot: Learning to follow instructions by watching gameplay videos},
author={Cai, Shaofei and Zhang, Bowei and Wang, Zihao and Ma, Xiaojian and Liu, Anji and Liang, Yitao},
journal={arXiv preprint arXiv:2310.08235},
year={2023}
}