Skip to content

[ICCV 2023] StreamPETR: Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detection

License

Notifications You must be signed in to change notification settings

exiawsh/StreamPETR

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

81 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

StreamPETR

[ICCV2023] Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detection

PWC PWC arXiv


Introduction

This repository is an official implementation of StreamPETR.

News

  • [2023/07/14] StreamPETR is accepted by ICCV 2023.
  • [2023/05/03] StreamPETR-Large is the first online multi-view method that achieves comparable performance (62.0 mAP, 67.6 NDS and 65.3 AMOTA) with the baseline of lidar-based method.

Getting Started

Please follow our documentation step by step. If you like our work, please recommend it to your colleagues and friends.

  1. Environment Setup.
  2. Data Preparation.
  3. Training and Inference.

Model Zoo


Results on NuScenes Val Set.

Model Setting Pretrain Lr Schd Training Time NDS mAP FPS-pytorch Config Download
RepDETR3D EVA02-L - 900q EVA02-L 24ep 12 hours (A100) 60.8 52.1 - config model
StreamPETR V2-99 - 900q FCOS3D 24ep 13 hours 57.1 48.2 12.5 config model/log
RepDETR3D V2-99 - 900q FCOS3D 24ep 13 hours 58.4 50.1 13.1 config model/log
StreamPETR R50 - 900q ImageNet 90ep 36 hours 53.7 43.2 26.7 config model/log
StreamPETR R50 - 428q NuImg 60ep 26 hours 54.6 44.9 31.7 config model/log

The detailed results can be found in the training log. For other results on nuScenes val set, please see Here. Notes:

  • FPS is measured on NVIDIA RTX 3090 GPU with batch size of 1 (containing 6 view images, without using flash attention) and FP32.
  • The training time is measured with 8x 2080ti GPUs.
  • RepDETR3D uses deformable attention, which is inspired by DETR3D and Sparse4D.

Results on NuScenes Test Set.

Model Setting Pretrain NDS mAP AMOTA AMOTP
StreamPETR V2-99 - 900q DD3D 63.6 55.0 - -
StreamPETR ViT-Large-900q - 67.6 62.0 65.3 87.6

Currently Supported Features

  • StreamPETR code (also including PETR and Focal-PETR)
  • Flash attention
  • Deformable attention (RepDETR3D)
  • Checkpoints
  • Sliding window training
  • Efficient training in streaming video
  • TensorRT inference
  • 3D object tracking

Acknowledgements

We thank these great works and open-source codebases:

Citation

If you find StreamPETR is useful in your research or applications, please consider giving us a star 🌟 and citing it by the following BibTeX entry.

@article{wang2023exploring,
  title={Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detection},
  author={Wang, Shihao and Liu, Yingfei and Wang, Tiancai and Li, Ying and Zhang, Xiangyu},
  journal={arXiv preprint arXiv:2303.11926},
  year={2023}
}

About

[ICCV 2023] StreamPETR: Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detection

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published