Official repository for Find n' Propagate: Open-Vocabulary 3D Object Detection in Urban Environments.
Highlights:
Find n' Propagate
has been accepted at ECCV 2024!
Use transfusion_lidar.yaml config for testing, need the normal class names so that OpenPCDet loads all the GT classes in the correct order.
Method | Arch | VLM | mAP | NDS | AP_B | AP_N | AR_N | Download |
---|---|---|---|---|---|---|---|---|
Ours | Transfusion | GLIP | 44.95 | 47.87 | 52.48 | 33.65 | 58.46 | model 94mb |
Method | Arch | VLM | mAP | NDS | AP_B | AP_N | AR_N | Download |
---|---|---|---|---|---|---|---|---|
Ours | Transfusion | GLIP | 31.44 | 34.53 | 67.41 | 16.03 | 49.78 | model 94mb |
Please refer to INSTALL.md for the installation of OpenPCDet
.
Please refer to GETTING_STARTED.md to get started with OpenPCDet
.
Download nuscenes_infos_train_mono3d.coco.json and nuscenes_glip_train_pred.pth to OpenPCDet/data/training_pred. This will be loaded by the PreprocessedGLIP class in pcdet/models/preprocessed_detector.py to generate pseudo-labels.
- Extract
Greedy Box Seeker
Boxes
python extract_pseudo_labels.py --cfg_file cfgs/nuscenes_box_seeker_proposals.yaml --folder ../data/pseudo_labels/nuscenes_box_seeker_proposals/
- Run Self-training
python train_st.py --cfg_file tools/cfgs/transfusion_lidar_st_nodisaugs_0.1sample_nomaxst_60confqueue_rotw_novelw_3sepinf_10fixcp_0.5unkw_1.0trans_0.785rot_0.2drop_glip_cpstonly_0.9997mom.yaml
- Evaluate on all classes
python test.py --cfg_file cfgs/nuscenes_models/transfusion_lidar.yaml --ckpt ../output/transfusion_lidar_st_nodisaugs_0.1sample_nomaxst_60confqueue_rotw_novelw_3sepinf_10fixcp_0.5unkw_1.0trans_0.785rot_0.2drop_glip_cpstonly_0.9997mom/default/ckpt/checkpoint_epoch_20.pth
OpenPCDet
is released under the Apache 2.0 license.
OpenPCDet
is an open source project for LiDAR-based 3D scene perception that supports multiple
LiDAR-based perception models as shown above. Some parts of PCDet
are learned from the official released codes of the above supported methods.
We would like to thank for their proposed methods and the official implementation.
We hope that this repo could serve as a strong and flexible codebase to benefit the research community by speeding up the process of reimplementing previous works and/or developing new methods.
If you find this project useful in your research, please consider citing:
@misc{openpcdet2020,
title={OpenPCDet: An Open-source Toolbox for 3D Object Detection from Point Clouds},
author={OpenPCDet Development Team},
howpublished = {\url{https://github.com/open-mmlab/OpenPCDet}},
year={2020}
}
@inproceedings{findnpropagate2024,
title={Find n' Propagate: Open Vocabulary 3D Object Detection in Urban Scenes},
author={Djamahl Etchegaray and Zi Huang and Tatsuya Harada and Yadan Luo},
title = {European Conference on Computer Vision (ECCV)},
year = {2024},
publisher = {Springer},
year={2024},
volume={}, # TODO: fill
pages={},
}