This repository contains the code for SC-Explorer, our framework for incremental fusion of 3D semantic scene completion and safe and efficient integration thereof into mobile robotic mapping and planning for exploration.
Incremental 3D scene completion for safe and efficient exploration mapping and planning.
Credits
Setup
Examples
If you find this useful for your research, please consider citing our paper:
- Lukas Schmid, Mansoor Nasir Cheema, Victor Reijgwart, Roland Siegwart, Federico Tombari, and Cesar Cadena, "SC-Explorer: Incremental 3D Scene Completion for Safe and Efficient Exploration Mapping and Planning" in ArXiv Preprint, 2022.
[ ArXiv | Video]
@article{schmid2022scexplorer, title={SC-Explorer: Incremental 3D Scene Completion for Safe and Efficient Exploration Mapping and Planning}, author={Schmid, Lukas and Cheema, Mansoor Nasir and Reijgwart, Victor and Siegwart, Roland and Tombari, Federico and Cadena, Cesar}, journal={arXiv preprint arXiv:2208.08307}, year={2022} }
An overview of SC-Explorer is available on YouTube:
ℹ️ Note
The code is provided on an 'as-is' basis. While everything should work and is tested, photorealistic simulation and GPU inference can be cumbersome and we cannot provide support for setting this up.
-
Install ROS (Desktop full recommended) if not already done so.
-
Install system dependencies:
sudo apt install python-wstool python-catkin-tools ros-$ROS_DISTRO-cmake-modules ros-$ROS_DISTRO-control-toolbox ros-$ROS_DISTRO-joy ros-$ROS_DISTRO-octomap-ros ros-$ROS_DISTRO-geographic-msgs autoconf libyaml-cpp-dev protobuf-compiler libgoogle-glog-dev liblapacke-dev libgeographic-dev
- Setup catkin workspace using catkin-tools
mkdir -p ~/catkin_ws/src
cd ~/catkin_ws
catkin init
catkin config --extend /opt/ros/$ROS_DISTRO
catkin config --cmake-args -DCMAKE_BUILD_TYPE=Release
catkin config --merge-devel
- Checkout the GitHub repository:
cd ~/catkin_ws/src
git clone [email protected]:ethz-asl/ssc_exploration.git # Requires Git SSH
- Install dependencies
wstool init . ./ssc_exploration/.rosinstall # Requires Git SSH
wstool update
- Compile
catkin build ssc_planning
- Build the scene completion network based on PAL-Net following the instructions in
ssc_network/
.
The following tools were used in our experiments, other simulation or real robot setups should work, too:
-
Unreal Engine Setup Unreal Engine 4.25.6 (UE4).
-
Airsim AirSim is a simulation software for simulating a MAV in Unreal Engine. We use the Airsim 1.2 plugin for UE4, which is provided by unreal_airsim below.
-
unreal_airsim unreal_airsim is a ROS interface to the simulated MAV in Unreal Engine for accessing odometry and sending trajectory commands.
-
Dataset The environment used in our experiments, including all plugins and ground truth can be downloaded here. Note that we can not guarantee compatibility with operating systems other than Ubuntu or different versions of AirSim/UE4.
Instructions to train and test the SC-Network as well as pretrained weights are given in ssc_network/
.
- Start Unreal Engine:
cd <UNREAL_INSTALL_DIR> # Move to the unreal install directory
./Engine/Binaries/Linux/UE4Editor your-project-file.uproject -opengl4
- Start the SSC Network (Make sure the network is setup as explained in
ssc_network/
first):
export SSC_DIR=/home/$USER/catkin_ws/src/ssc_exploration/ssc_network
python3 $SSC_DIR/infer_ros.py --model palnet --resume $SSC_DIR/pretrained_models/weights/PALNet.pth.tar
- Launch planning pipeline with the desired planning configuration specified as argument:
roslaunch ssc_planning run.launch planner_config_file:=sc_explorer.yaml output_directory:=path/to/output
Feel free to play with parameters in sc_explorer.yaml
or use different gain evaluators by specifying them in the config. Note that all configs are composed of two files, where baseline
specifies the shared parameters and sc_explorer
or exploration
provide specialized parameters for view planning.
Running the pipeline as explained above will write all output files into a timestamped folder in path/to/output
. To evaluate the recorded data and visualize the map, run:
python3 ./ssc_mapping/src/eval/eval_plots.py [output_dir] [gt_file_path] [eval_type]
Where output_dir
should point to the previously created data directory, gt_file_path
points to the ground truth (available here for our simulated scene), and eval_type
is one of tsdf
or hierarchical
to only evaluate measured or also scene completed areas, respectively.