Code of Integrally Migrating Pre-trained Transformer Encoder-decoders for Visual Object Detection.
The code is based on mmdetection, please refer to get_started.md and MMDET_README.md to set up the environment and prepare the data.
We provide 9 configuration files in the configs directory.
| Config File | Backbone | Epochs | Box AP | Mask AP | Download |
|---|---|---|---|---|---|
| imted_faster_rcnn_vit_small_3x_coco | ViT-S | 36 | 48.2 | model | |
| imted_faster_rcnn_vit_base_3x_coco | ViT-B | 36 | 52.9 | model | |
| imted_faster_rcnn_vit_large_3x_coco | ViT-L | 36 | 55.4 | model | |
| imted_mask_rcnn_vit_small_3x_coco | ViT-S | 36 | 48.7 | 42.7 | model |
| imted_mask_rcnn_vit_base_3x_coco | ViT-B | 36 | 53.3 | 46.4 | model |
| imted_mask_rcnn_vit_large_3x_coco | ViT-L | 36 | 55.5 | 48.1 | model |
| imted_faster_rcnn_vit_base_2x_base_training_coco | ViT-B | 24 | 50.6 | model | |
| imted_faster_rcnn_vit_base_2x_finetuning_10shot_coco | ViT-B | 108 | 23.0 | model | |
| imted_faster_rcnn_vit_base_2x_finetuning_30shot_coco | ViT-B | 108 | 30.4 | model |
The pre-trained model is trained with the official MAE code. For ViT-S, we use a 4-layer decoder with dimension 256 for 800 epochs of pre-training. For ViT-B, we use an 8-layer decoder with dimension 512 for 1600 epochs of pre-training. Pre-trained weights can be downloaded from the official MAE weight. For ViT-L, we use an 8-layer decoder with dimension 512 for 1600 epochs of pre-training. Pre-trained weights can be downloaded from the official MAE weight.
For all experiments, remember to modify the path of pre-trained weights in the configuration files, e.g. configs/imted/imted_faster_rcnn_vit_small_3x_coco.py.
For few-shot experiments, please refer to FsDet for data preparation. Remember to modify the path of json in the configuration files, e.g. configs/imted/few_shot/imted_faster_rcnn_vit_base_2x_base_training_coco.py.
tools/dist_test.sh "path/to/config/file.py" "path/to/trained/weights.pth" 1 --eval bboxtools/dist_train.sh "path/to/config/file.py" 8 If you have any questions with our code/paper, feel free to contact me through issue or e-mail.