This repository is an implementation of StreamPETR with 3dppe.
-
Prepare nuScenes dataset and generate 2D annotations and temporal information for training & evaluation. (see streamPETR)
-
Conda env
conda create -n xxx python=3.8 -y
conda activate xxx
pip install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html
pip install flash-attn==0.2.2 # (Tesla v100 is not compatible)
pip install mmcv-full==1.6.0
pip install mmdet==2.28.2
pip install mmsegmentation==0.30.0
git clone https://github.com/open-mmlab/mmdetection3d.git
cd mmdetection3d
git checkout v1.0.0rc6
pip install -v -e .Note : make sure that
numba 0.53.0 numpy 1.23.5
(if not, reinstall numba==0.53.0).
Catalogue:
tree -d -L 1
.
├── ckpts
├── data
├── mmdetection3d
├── projects
└── tools
- Train & Infer
tools/dist_train.sh [-config] [-num_gpus]tools/dist_test.sh [-config] [-model] [-num_gpus] --eval bbox| Model | Setting | Pretrain | Lr Schd | Training Time | NDS | mAP | Config | Download |
|---|---|---|---|---|---|---|---|---|
| StreamPETR | V2-99-900q-800x320 | FCOS3D | 24ep | 13h | 57.1 | 48.3 | config | model/log |
| Stream3dppe | V2-99-900q-800x320 | FCOS3D | 24ep | 16h | 58.45/58.45 | 49.95/50.04 | config | （model1,model2)/(log1,log2) |
| Stream3dppe_gt_detph | V2-99-900q-800x320 | FCOS3D | 24ep | 22h | 61.7 | 55.3 | config | model/log |
| StreamPETR | V2-99-900q-1600x640 | FCOS3D | 24ep | |||||
| Stream3DPPE | V2-99-900q-1600x640 | FCOS3D | 24ep |
Note : Stream3dppe is trained on 4 x RTX 3090 with bs4 ,while Stream3dppe_gt_detph is trained on 4 x RTX 2080Ti with bs2 .
More result please refer to https://github.com/drilistbox/3DPPE.
Many thanks to the authors of PETR and StreamPETR.
If you find this project useful for your research, please consider citing:
@article{shu20233DPPE,
title={3DPPE: 3D Point Positional Encoding for Multi-Camera 3D Object Detection Transformers},
author={Shu, Changyong and Deng, Jiajun and Yu, Fisher and Liu, Yifan},
journal={arXiv preprint arXiv:2211.14710},
year={2023}
}