Dual-Radar (provided by 'ADEPTLab') is a brand new dataset based on 4D radar that can be used for studies on deep learning object detection and tracking in the field of autonomous driving. The system of ego vehicle includes a high-resolution camera, a 80-line LiDAR and two up-to-date and different models of 4D radars operating in different modes(Arbe and ARS548). The dataset comprises of raw data collected from ego vehicle, including scenarios such as tunnels and urban, with weather conditions rainy, cloudy and sunny. Our dataset also includes data from different time periods, including dusk, nighttime, and daytime. Our collected raw data amounts to a total of 12.5 hours, encompassing a total distance of over 600 kilometers. Our dataset covers a route distance of approximately 50 kilometers. It consists of 151 continuous time sequences, with the majority being 20-second sequences, resulting in a total of 10,007 carefully time-synchronized frames.

a) First-person perspective observation

b) Third-person perspective observation

Figure 1. Up to a visual range of 80 meters in urban

• Notice: On the left is a color RGB image, while on the right side, the cyan represents the Arbe point cloud, the white represents the LiDAR point cloud, and the yellow represents the ARS548 point cloud.

a) sunny,daytime,up to a distance of 80 meters	b) sunny,nightime,up to a distance of 80 meters
c) rainy,daytime,up to a distance of 80 meters	d) cloudy,daytime,up to a distance of 80 meters

Figure 1. Up to a visual range of 80 meters in urban

Figure 2. Data Visualization

The URLs listed below are useful for using the Dual-Radar dataset and benchmark:

Figure 3. Sensor Configuration and Coordinate Systems

• The specification of the autonomous vehicle system platform

• The statistics of number of points cloud per frame

We separately counted the number of instances for each category in the Dual-Radar dataset and the distribution of different types of weather.

Figure 4. Distribution of weather conditions.

About two-thirds of our data were collected under normal weather conditions, and about one-third were collected under rainy and cloudy conditions. We collected 577 frames in rainy weather, which is about 5.5% of the total dataset. The rainy weather data we collect can be used to test the performance of different 4D radars in adverse weather conditions.

Figure 5. Distribution of instance conditions.

This is the documentation for how to use our detection frameworks with Dual-Radar dataset. We tested the Dual-Radar detection frameworks on the following environment:

• Python 3.8.16 (3.10+ does not support open3d.)
• Ubuntu 18.04/20.04
• Torch 1.10.1+cu113
• CUDA 11.3
• opencv 4.2.0.32

[2022-09-30] The dataset will provide a link for access to the data for further research as soon as possible.

1. After all files are downloaded, please arrange the workspace directory with the following structure:

Organize your code structure as follows

Frameworks
  ├── checkpoints
  ├── data
  ├── docs
  ├── Dual-Radardet
  ├── output

Organize the dataset according to the following file structure

Dataset
  ├── ImageSets
        ├── training.txt
        ├── trainingval.txt
        ├── val.txt
        ├── testing.txt
  ├── training
        ├── arbe
        ├── ars548
        ├── calib
        ├── image_2
        ├── label_2
        ├── velodyne
  ├── testing
        ├── arbe
        ├── ars548
        ├── calib
        ├── image_2
        ├── velodyne

1. Clone the repository

 git clone https://github.com/adept-thu/Dual-Radar.git
 cd Dual-Radar

2. Create a conda environment

conda create -n Dual-Radardet python=3.8.16
conda activate Dual-Radardet

3. Install PyTorch (We recommend pytorch 1.10.1.)

4. Install the dependencies

pip install -r requirements.txt

5. Install Spconv（our cuda version is 113）

pip install spconv-cu113

6. Build packages for Dual-Radardet

python setup.py develop

• Generate the data infos by running the following command:

python -m pcdet.datasets.mine.kitti_dataset create_mine_infos tools/cfgs/dataset_configs/mine_dataset.yaml
# or you want to use arbe data
python -m pcdet.datasets.mine.kitti_dataset_arbe create_mine_infos tools/cfgs/dataset_configs/mine_dataset_arbe.yaml
# or ars548
python -m pcdet.datasets.mine.kitti_dataset_ars548 create_mine_infos tools/cfgs/dataset_configs/mine_dataset_ars548.yaml

• To train the model on single GPU, prepare the total dataset and run

python train.py --cfg_file ${CONFIG_FILE}

• To train the model on multi-GPUS, prepare the total dataset and run

sh scripts/dist_train.sh ${NUM_GPUS} --cfg_file ${CONFIG_FILE}

• To evaluate the model on single GPU, modify the path and run

python test.py --cfg_file ${CONFIG_FILE} --batch_size ${BATCH_SIZE} --ckpt ${CKPT}

• To evaluate the model on multi-GPUS, modify the path and run

sh scripts/dist_test.sh ${NUM_GPUS} \
    --cfg_file ${CONFIG_FILE} --batch_size ${BATCH_SIZE}

Here we provide a quick demo to test a pretrained model on the custom point cloud data and visualize the predicted results

• Download the pretrained model as shown in Table 4~8.
• Make sure you have installed the Open3d and mayavi visualization tools. If not, you could install it as follow:

pip install open3d
pip install mayavi

• prepare your point cloud data

points[:, 3] = 0 
np.save(`my_data.npy`, points)

• Run the demo with a pretrained model and point cloud data as follows

python demo.py --cfg_file ${CONFIG_FILE} \
    --ckpt ${CKPT} \
    --data_path ${POINT_CLOUD_DATA}

The Dual-Radar dataset is published under the CC BY-NC-ND License, and all codes are published under the Apache License 2.0.