Bump version to v3.2.0 (#11030)

.circleci/test.yml

@@ -94,21 +94,27 @@ jobs:
  type: string
  cuda:
  type: enum
- enum: ["11.1", "11.7"]
+ enum: ["11.1", "11.7", "11.8"]
  cudnn:
  type: integer
  default: 8
  machine:
- image: ubuntu-2004-cuda-11.4:202110-01
+ image: linux-cuda-11:default
  # docker_layer_caching: true
- resource_class: gpu.nvidia.small
+ resource_class: gpu.nvidia.small.multi
  steps:
  - checkout
  - run:
  # CLoning repos in VM since Docker doesn't have access to the private key
  name: Clone Repos
  command: |
  git clone -b main --depth 1 ssh:https://[email protected]/open-mmlab/mmengine.git /home/circleci/mmengine
+ - run:
+ name: Install nvidia-container-toolkit and Restart Docker
+ command: |
+ sudo apt-get update
+ sudo apt-get install -y nvidia-container-toolkit
+ sudo systemctl restart docker
  - run:
  name: Build Docker image
  command: |

.dev_scripts/benchmark_train_models.txt

@@ -2,11 +2,13 @@ atss/atss_r50_fpn_1x_coco.py
 faster_rcnn/faster-rcnn_r50_fpn_1x_coco.py
 mask_rcnn/mask-rcnn_r50_fpn_1x_coco.py
 cascade_rcnn/cascade-mask-rcnn_r50_fpn_1x_coco.py
+configs/grounding_dino/grounding_dino_swin-t_finetune_16xb2_1x_coco.py
+configs/glip/glip_atss_swin-t_a_fpn_dyhead_16xb2_ms-2x_funtune_coco.py
+configs/ddq/ddq-detr-4scale_r50_8xb2-12e_coco.py
 panoptic_fpn/panoptic-fpn_r50_fpn_1x_coco.py
 retinanet/retinanet_r50_fpn_1x_coco.py
 rtmdet/rtmdet_s_8xb32-300e_coco.py
 rtmdet/rtmdet-ins_s_8xb32-300e_coco.py
-deformable_detr/deformable-detr_r50_16xb2-50e_coco.py
 fcos/fcos_r50-caffe_fpn_gn-head-center-normbbox-centeronreg-giou_1x_coco.py
 centernet/centernet-update_r50-caffe_fpn_ms-1x_coco.py
 dino/dino-4scale_r50_8xb2-12e_coco.py

.dev_scripts/gather_models.py

@@ -12,7 +12,7 @@
 import yaml
 from mmengine.config import Config
 from mmengine.fileio import dump
-from mmengine.utils import mkdir_or_exist, scandir
+from mmengine.utils import digit_version, mkdir_or_exist, scandir
 
 
 def ordered_yaml_dump(data, stream=None, Dumper=yaml.SafeDumper, **kwds):
@@ -45,7 +45,7 @@ def process_checkpoint(in_file, out_file):
 
  # if it is necessary to remove some sensitive data in checkpoint['meta'],
  # add the code here.
- if torch.__version__ >= '1.6':
+ if digit_version(torch.__version__) >= digit_version('1.6'):
  torch.save(checkpoint, out_file, _use_new_zipfile_serialization=False)
  else:
  torch.save(checkpoint, out_file)

README.md

@@ -25,6 +25,7 @@
 [![license](https://img.shields.io/github/license/open-mmlab/mmdetection.svg)](https://github.com/open-mmlab/mmdetection/blob/main/LICENSE)
 [![open issues](https://isitmaintained.com/badge/open/open-mmlab/mmdetection.svg)](https://github.com/open-mmlab/mmdetection/issues)
 [![issue resolution](https://isitmaintained.com/badge/resolution/open-mmlab/mmdetection.svg)](https://github.com/open-mmlab/mmdetection/issues)
+[![Open in OpenXLab](https://cdn-static.openxlab.org.cn/app-center/openxlab_demo.svg)](https://openxlab.org.cn/apps?search=mmdet)
 
 [📘Documentation](https://mmdetection.readthedocs.io/en/latest/) |
 [🛠️Installation](https://mmdetection.readthedocs.io/en/latest/get_started.html) |
@@ -102,6 +103,52 @@ Apart from MMDetection, we also released [MMEngine](https://github.com/open-mmla
 
 ### Highlight
 
+**v3.2.0** was released in 12/10/2023:
+
+**1. Detection Transformer SOTA Model Collection**
+(1) Supported four updated and stronger SOTA Transformer models: [DDQ](configs/ddq/README.md), [CO-DETR](projects/CO-DETR/README.md), [AlignDETR](projects/AlignDETR/README.md), and [H-DINO](projects/HDINO/README.md).
+(2) Based on CO-DETR, MMDet released a model with a COCO performance of 64.1 mAP.
+(3) Algorithms such as DINO support `AMP/Checkpoint/FrozenBN`, which can effectively reduce memory usage.
+
+**2. [Comprehensive Performance Comparison between CNN and Transformer](<(projects/RF100-Benchmark/README.md)>)**
+RF100 consists of a dataset collection of 100 real-world datasets, including 7 domains. It can be used to assess the performance differences of Transformer models like DINO and CNN-based algorithms under different scenarios and data volumes. Users can utilize this benchmark to quickly evaluate the robustness of their algorithms in various scenarios.
+
+<div align=center>
+<img src="https://github.com/open-mmlab/mmdetection/assets/17425982/86420903-36a8-410d-9251-4304b9704f7d"/>
+</div>
+
+**3. Support for [GLIP](configs/glip/README.md) and [Grounding DINO](configs/grounding_dino/README.md) fine-tuning, the only algorithm library that supports Grounding DINO fine-tuning**
+The Grounding DINO algorithm in MMDet is the only library that supports fine-tuning. Its performance is one point higher than the official version, and of course, GLIP also outperforms the official version.
+We also provide a detailed process for training and evaluating Grounding DINO on custom datasets. Everyone is welcome to give it a try.
+
+| Model | Backbone | Style | COCO mAP | Official COCO mAP |
+| :----------------: | :------: | :-------: | :--------: | :---------------: |
+| Grounding DINO-T | Swin-T | Zero-shot | 48.5 | 48.4 |
+| Grounding DINO-T | Swin-T | Finetune | 58.1(+0.9) | 57.2 |
+| Grounding DINO-B | Swin-B | Zero-shot | 56.9 | 56.7 |
+| Grounding DINO-B | Swin-B | Finetune | 59.7 | |
+| Grounding DINO-R50 | R50 | Scratch | 48.9(+0.8) | 48.1 |
+
+**4. Support for the open-vocabulary detection algorithm [Detic](projects/Detic_new/README.md) and multi-dataset joint training.**
+**5. Training detection models using [FSDP and DeepSpeed](<(projects/example_largemodel/README.md)>).**
+
+| ID | AMP | GC of Backbone | GC of Encoder | FSDP | Peak Mem (GB) | Iter Time (s) |
+| :-: | :-: | :------------: | :-----------: | :--: | :-----------: | :-----------: |
+| 1 | | | | | 49 (A100) | 0.9 |
+| 2 | √ | | | | 39 (A100) | 1.2 |
+| 3 | | √ | | | 33 (A100) | 1.1 |
+| 4 | √ | √ | | | 25 (A100) | 1.3 |
+| 5 | | √ | √ | | 18 | 2.2 |
+| 6 | √ | √ | √ | | 13 | 1.6 |
+| 7 | | √ | √ | √ | 14 | 2.9 |
+| 8 | √ | √ | √ | √ | 8.5 | 2.4 |
+
+**6. Support for the [V3Det](configs/v3det/README.md) dataset, a large-scale detection dataset with over 13,000 categories.**
+
+<div align=center>
+ <img width=960 src="https://github.com/open-mmlab/mmdetection/assets/17425982/9c216387-02be-46e6-b0f2-b856f80f6d84"/>
+</div>
+
 We are excited to announce our latest work on real-time object recognition tasks, **RTMDet**, a family of fully convolutional single-stage detectors. RTMDet not only achieves the best parameter-accuracy trade-off on object detection from tiny to extra-large model sizes but also obtains new state-of-the-art performance on instance segmentation and rotated object detection tasks. Details can be found in the [technical report](https://arxiv.org/abs/2212.07784). Pre-trained models are [here](configs/rtmdet).
 
 [![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/rtmdet-an-empirical-study-of-designing-real/real-time-instance-segmentation-on-mscoco)](https://paperswithcode.com/sota/real-time-instance-segmentation-on-mscoco?p=rtmdet-an-empirical-study-of-designing-real)
@@ -118,13 +165,6 @@ We are excited to announce our latest work on real-time object recognition tasks
 <img src="https://user-images.githubusercontent.com/12907710/208044554-1e8de6b5-48d8-44e4-a7b5-75076c7ebb71.png"/>
 </div>
 
-**v3.1.0** was released in 30/6/2023:
-
-- Supports tracking algorithms including multi-object tracking (MOT) algorithms SORT, DeepSORT, StrongSORT, OCSORT, ByteTrack, QDTrack, and video instance segmentation (VIS) algorithm MaskTrackRCNN, Mask2Former-VIS.
-- Support [ViTDet](projects/ViTDet)
-- Supports inference and evaluation of multimodal algorithms [GLIP](configs/glip) and [XDecoder](projects/XDecoder), and also supports datasets such as COCO semantic segmentation, COCO Caption, ADE20k general segmentation, and RefCOCO. GLIP fine-tuning will be supported in the future.
-- Provides a [gradio demo](https://github.com/open-mmlab/mmdetection/blob/dev-3.x/projects/gradio_demo/README.md) for image type tasks of MMDetection, making it easy for users to experience.
-
 ## Installation
 
 Please refer to [Installation](https://mmdetection.readthedocs.io/en/latest/get_started.html) for installation instructions.
@@ -236,9 +276,12 @@ Results and models are available in the [model zoo](docs/en/model_zoo.md).
   <li><a href="configs/dab_detr">DAB-DETR (ICLR'2022)</a></li>
   <li><a href="configs/dino">DINO (ICLR'2023)</a></li>
   <li><a href="configs/glip">GLIP (CVPR'2022)</a></li>
+  <li><a href="configs/ddq">DDQ (CVPR'2023)</a></li>
   <li><a href="projects/DiffusionDet">DiffusionDet (ArXiv'2023)</a></li>
   <li><a href="projects/EfficientDet">EfficientDet (CVPR'2020)</a></li>
+  <li><a href="projects/ViTDet">ViTDet (ECCV'2022)</a></li>
   <li><a href="projects/Detic">Detic (ECCV'2022)</a></li>
+  <li><a href="projects/CO-DETR">CO-DETR (ICCV'2023)</a></li>
   </ul>
   </td>
   <td>
@@ -260,13 +303,15 @@ Results and models are available in the [model zoo](docs/en/model_zoo.md).
   <li><a href="projects/SparseInst">SparseInst (CVPR'2022)</a></li>
   <li><a href="configs/rtmdet">RTMDet (ArXiv'2022)</a></li>
   <li><a href="configs/boxinst">BoxInst (CVPR'2021)</a></li>
+  <li><a href="projects/ConvNeXt-V2">ConvNeXt-V2 (Arxiv'2023)</a></li>
   </ul>
   </td>
   <td>
   <ul>
   <li><a href="configs/panoptic_fpn">Panoptic FPN (CVPR'2019)</a></li>
   <li><a href="configs/maskformer">MaskFormer (NeurIPS'2021)</a></li>
   <li><a href="configs/mask2former">Mask2Former (ArXiv'2021)</a></li>
+  <li><a href="configs/XDecoder">XDecoder (CVPR'2023)</a></li>
   </ul>
   </td>
   <td>

README_zh-CN.md

@@ -25,6 +25,7 @@
 [![license](https://img.shields.io/github/license/open-mmlab/mmdetection.svg)](https://github.com/open-mmlab/mmdetection/blob/main/LICENSE)
 [![open issues](https://isitmaintained.com/badge/open/open-mmlab/mmdetection.svg)](https://github.com/open-mmlab/mmdetection/issues)
 [![issue resolution](https://isitmaintained.com/badge/resolution/open-mmlab/mmdetection.svg)](https://github.com/open-mmlab/mmdetection/issues)
+[![Open in OpenXLab](https://cdn-static.openxlab.org.cn/app-center/openxlab_demo.svg)](https://openxlab.org.cn/apps?search=mmdet)
 
 [📘使用文档](https://mmdetection.readthedocs.io/zh_CN/latest/) |
 [🛠️安装教程](https://mmdetection.readthedocs.io/zh_CN/latest/get_started.html) |
@@ -101,6 +102,53 @@ MMDetection 是一个基于 PyTorch 的目标检测开源工具箱。它是 [Ope
 
 ### 亮点
 
+**v3.2.0** 版本已经在 2023.10.12 发布：
+
+**1. 检测 Transformer SOTA 模型大合集**
+(1) 支持了 [DDQ](configs/ddq/README.md)、[CO-DETR](projects/CO-DETR/README.md)、[AlignDETR](projects/AlignDETR/README.md) 和 [H-DINO](projects/HDINO/README.md) 4 个更新更强的 SOTA Transformer 模型
+(2) 基于 CO-DETR, MMDet 中发布了 COCO 性能为 64.1 mAP 的模型
+(3) DINO 等算法支持 AMP/Checkpoint/FrozenBN，可以有效降低显存
+
+**2. [提供了全面的 CNN 和 Transformer 的性能对比](projects/RF100-Benchmark/README_zh-CN.md)**
+RF100 是由 100 个现实收集的数据集组成，包括 7 个域，可以验证 DINO 等 Transformer 模型和 CNN 类算法在不同场景不同数据量下的性能差异。用户可以用这个 Benchmark 快速验证自己的算法在不同场景下的鲁棒性。
+
+<div align=center>
+<img src="https://github.com/open-mmlab/mmdetection/assets/17425982/86420903-36a8-410d-9251-4304b9704f7d"/>
+</div>
+
+**3. 支持了 [GLIP](configs/glip/README.md) 和 [Grounding DINO](configs/grounding_dino/README.md) 微调，全网唯一支持 Grounding DINO 微调**
+MMDet 中的 Grounding DINO 是全网唯一支持微调的算法库，且性能高于官方 1 个点，当然 GLIP 也比官方高。
+我们还提供了详细的 Grounding DINO 在自定义数据集上训练评估的流程，欢迎大家试用。
+
+| Model | Backbone | Style | COCO mAP | Official COCO mAP |
+| :----------------: | :------: | :-------: | :--------: | :---------------: |
+| Grounding DINO-T | Swin-T | Zero-shot | 48.5 | 48.4 |
+| Grounding DINO-T | Swin-T | Finetune | 58.1(+0.9) | 57.2 |
+| Grounding DINO-B | Swin-B | Zero-shot | 56.9 | 56.7 |
+| Grounding DINO-B | Swin-B | Finetune | 59.7 | |
+| Grounding DINO-R50 | R50 | Scratch | 48.9(+0.8) | 48.1 |
+
+**4. 支持开放词汇检测算法 [Detic](projects/Detic_new/README.md) 并提供多数据集联合训练可能**
+
+**5. 轻松使用 [FSDP 和 DeepSpeed 训练检测模型](projects/example_largemodel/README_zh-CN.md)**
+
+| ID | AMP | GC of Backbone | GC of Encoder | FSDP | Peak Mem (GB) | Iter Time (s) |
+| :-: | :-: | :------------: | :-----------: | :--: | :-----------: | :-----------: |
+| 1 | | | | | 49 (A100) | 0.9 |
+| 2 | √ | | | | 39 (A100) | 1.2 |
+| 3 | | √ | | | 33 (A100) | 1.1 |
+| 4 | √ | √ | | | 25 (A100) | 1.3 |
+| 5 | | √ | √ | | 18 | 2.2 |
+| 6 | √ | √ | √ | | 13 | 1.6 |
+| 7 | | √ | √ | √ | 14 | 2.9 |
+| 8 | √ | √ | √ | √ | 8.5 | 2.4 |
+
+**6. 支持了 [V3Det](configs/v3det/README.md) 1.3w+ 类别的超大词汇检测数据集**
+
+<div align=center>
+ <img width=960 src="https://github.com/open-mmlab/mmdetection/assets/17425982/9c216387-02be-46e6-b0f2-b856f80f6d84"/>
+</div>
+
 我们很高兴向大家介绍我们在实时目标识别任务方面的最新成果 RTMDet，包含了一系列的全卷积单阶段检测模型。 RTMDet 不仅在从 tiny 到 extra-large 尺寸的目标检测模型上实现了最佳的参数量和精度的平衡，而且在实时实例分割和旋转目标检测任务上取得了最先进的成果。 更多细节请参阅[技术报告](https://arxiv.org/abs/2212.07784)。 预训练模型可以在[这里](configs/rtmdet)找到。
 
 [![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/rtmdet-an-empirical-study-of-designing-real/real-time-instance-segmentation-on-mscoco)](https://paperswithcode.com/sota/real-time-instance-segmentation-on-mscoco?p=rtmdet-an-empirical-study-of-designing-real)
@@ -117,13 +165,6 @@ MMDetection 是一个基于 PyTorch 的目标检测开源工具箱。它是 [Ope
 <img src="https://user-images.githubusercontent.com/12907710/208044554-1e8de6b5-48d8-44e4-a7b5-75076c7ebb71.png"/>
 </div>
 
-**v3.1.0** 版本已经在 2023.6.30 发布：
-
-- 支持 Tracking 类算法，包括多目标跟踪 MOT 算法 SORT、DeepSORT、StrongSORT、OCSORT、ByteTrack、QDTrack 和视频实例分割 VIS 算法 MaskTrackRCNN、Mask2Former-VIS。
-- 支持 [ViTDet](projects/ViTDet)
-- 支持多模态开放检测算法 [GLIP](configs/glip) 和 [XDecoder](projects/XDecoder) 推理和评估，并同时支持了 COCO 语义分割、COCO Caption、ADE20k 通用分割、RefCOCO 等数据集。后续将支持 GLIP 微调
-- 提供了包括 MMDetection 图片任务的 [gradio demo](https://github.com/open-mmlab/mmdetection/blob/dev-3.x/projects/gradio_demo/README.md)，方便用户快速体验
-
 ## 安装
 
 请参考[快速入门文档](https://mmdetection.readthedocs.io/zh_CN/latest/get_started.html)进行安装。
@@ -237,9 +278,12 @@ MMDetection 是一个基于 PyTorch 的目标检测开源工具箱。它是 [Ope
   <li><a href="configs/dab_detr">DAB-DETR (ICLR'2022)</a></li>
   <li><a href="configs/dino">DINO (ICLR'2023)</a></li>
   <li><a href="configs/glip">GLIP (CVPR'2022)</a></li>
+  <li><a href="configs/ddq">DDQ (CVPR'2023)</a></li>
   <li><a href="projects/DiffusionDet">DiffusionDet (ArXiv'2023)</a></li>
   <li><a href="projects/EfficientDet">EfficientDet (CVPR'2020)</a></li>
+  <li><a href="projects/ViTDet">ViTDet (ECCV'2022)</a></li>
   <li><a href="projects/Detic">Detic (ECCV'2022)</a></li>
+  <li><a href="projects/CO-DETR">CO-DETR (ICCV'2023)</a></li>
   </ul>
   </td>
   <td>
@@ -261,13 +305,15 @@ MMDetection 是一个基于 PyTorch 的目标检测开源工具箱。它是 [Ope
   <li><a href="projects/SparseInst">SparseInst (CVPR'2022)</a></li>
   <li><a href="configs/rtmdet">RTMDet (ArXiv'2022)</a></li>
   <li><a href="configs/boxinst">BoxInst (CVPR'2021)</a></li>
+  <li><a href="projects/ConvNeXt-V2">ConvNeXt-V2 (Arxiv'2023)</a></li>
   </ul>
   </td>
   <td>
   <ul>
   <li><a href="configs/panoptic_fpn">Panoptic FPN (CVPR'2019)</a></li>
   <li><a href="configs/maskformer">MaskFormer (NeurIPS'2021)</a></li>
   <li><a href="configs/mask2former">Mask2Former (ArXiv'2021)</a></li>
+  <li><a href="configs/XDecoder">XDecoder (CVPR'2023)</a></li>
   </ul>
   </td>
   <td>
@@ -443,10 +489,10 @@ MMDetection 是一款由来自不同高校和企业的研发人员共同参与
 
 ## 欢迎加入 OpenMMLab 社区
 
-扫描下方的二维码可关注 OpenMMLab 团队的 [知乎官方账号](https://www.zhihu.com/people/openmmlab)，加入 OpenMMLab 团队的 [官方交流 QQ 群](https://jq.qq.com/?_wv=1027&k=aCvMxdr3)
+扫描下方的二维码可关注 OpenMMLab 团队的 [知乎官方账号](https://www.zhihu.com/people/openmmlab)，扫描下方微信二维码添加喵喵好友，进入 MMDectection 微信交流社群。【加好友申请格式：研究方向+地区+学校/公司+姓名】
 
 <div align="center">
-<img src="resources/zhihu_qrcode.jpg" height="400" /> <img src="resources/qq_group_qrcode.jpg" height="400" />
+<img src="resources/zhihu_qrcode.jpg" height="400" /> <img src="resources/miaomiao_qrcode.jpg" height="400" />
 </div>
 
 我们会在 OpenMMLab 社区为大家