To accelerate computer vision research and innovation for Vehicle-Infrastructure Cooperative Autonomous Driving (VICAD), we release DAIR-V2X Dataset, which is the first large-scale, multi-modality, multi-view dataset from real scenarios for VICAD. Furtherly, we introduce the Vehicle-Infrastructure Cooperative 3D Object Detection (VIC3D) task, which formulate the problem of cooperatively locating and identifying 3D objects using sensory inputs from both vehicle and infrastructure. To resolve the VIC3D object detection task and facilitate future research, we also introduce our VIC3D object detection benchmark as well as an extensive 3D detection benchmark for those who are interested in Single-View (SV) 3D detection tasks. In order to facilitate researchers to use the dataset and reproduce the entire benchmark, we open source the OpenDAIRV2X, towards serving VICAD research community.

• [2022.07] We release the OpenDAIRV2X codebase v1.0.0. The current version can faciliate the researchers to use the DAIR-V2X dataset and reproduce the benchmarks.
• [2022.03] Our Paper "DAIR-V2X: A Large-Scale Dataset for Vehicle-Infrastructure Cooperative 3D Object Detection" has been accepted by CVPR2022. Arxiv version could be seen here.
• [2022.02] DAIR-V2X dataset is availale here. It can be unlimitedly downloaded within mainland China.

• Support Train/Evaluation for VIC3D

  It will directly support model training and evaluation for VIC3D. Now the model inference and model training are mainly based on MMDetection3D, which is not quite convenient to carry the VICAD research.

• Evaluation (Model inference is based on MMDetection3D)

• Training based on MMDetection3D

• Direct Evaluation with DAIR-V2X Framework

• Direct Training with DAIR-V2X Framework

• Support different fusion methods for VIC3D

  It will directly support different fusion methods including early fusion/feature fusion/late fusion. Now it supports early fusion and late fusion.

• Early Fusion

• Feature Fusion

• Late Fusion

• Support multi-modality/single-modality detectors for VIC3D

  It will directly support different modaility detectors including image-modality detector, pointcloud-modality detector and image-pointcloud fusion detector. Now it supports image-modality detector ImvoxelNet, pointcloud-modality detector PointPillars.

• Image-modality

• Pointcloud-modality

• Multi-modality

• Support Cooperation-view/single-view detectors for VIC3D

  It directly supports different view's detectors for VIC3D, including infrastructure-view detector, vehicle-view detector, vehicle-infrastrucure cooperation-view detector.

• Infrastructure-view

• Vehicle-view

• Cooperation-view

VIC3D Benchmark is as following:

SV3D Benchmark based on DAIR-V2X-V is as following:

SV3D Benchmark based on DAIR-V2X-I is as following:

Please refer to getting_started.md for Installation, Evaluation, Benchmark and Training etc for VIC3D.

This project is not possible without the following codebases.

• MMDetection3D
• pypcd

If you find this project useful in your research, please consider cite:

@inproceedings{yu2022dairv2x,
    title={DAIR-V2X: A Large-Scale Dataset for Vehicle-Infrastructure Cooperative 3D Object Detection},
    author={Yu, Haibao and Luo, Yizhen and Shu, Mao and Huo, Yiyi and Yang, Zebang and Shi, Yifeng and Guo, Zhenglong and Li, Hanyu and Hu, Xing and Yuan, Jirui and Nie, Zaiqing},
    booktitle={IEEE/CVF Conf.~on Computer Vision and Pattern Recognition (CVPR)},
    month = jun,
    year={2022}
}

If any questions and suggenstations, please email to [email protected].