Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge MMSegmentation 1.x development branch dev-1.x to main branch 1.x for v1.0.0rc2 #2385

Merged
merged 59 commits into from
Dec 6, 2022
Merged
Changes from 1 commit
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
89f6647
[Doc] Updata transforms Doc
MengzhangLI Sep 19, 2022
5b5968d
fix typo
MengzhangLI Oct 8, 2022
124ec82
Merge pull request #2088 from MengzhangLI/transforms_doc
MeowZheng Nov 2, 2022
c2b4798
Update README.md
kitecats Nov 3, 2022
4d6d74e
Update README_zh-CN.md
kitecats Nov 3, 2022
7927591
Merge pull request #2259 from kitecats/dev-1.x
MeowZheng Nov 8, 2022
70daaaa
support padding in test and fix remove gt padding at post_process
xiexinch Nov 10, 2022
d0b35cd
fix get gt data
xiexinch Nov 10, 2022
d33af52
fix ut
xiexinch Nov 10, 2022
ef0f56d
Update visualization.md
matrixgame2018 Nov 12, 2022
2b854e6
Update visualization.md
matrixgame2018 Nov 13, 2022
921fdde
fix lint
xiexinch Nov 15, 2022
a39b16f
Merge pull request #2308 from xiexinch/fix_lint
MeowZheng Nov 15, 2022
26e9729
fix swin load statedict
xiexinch Nov 15, 2022
a06bf4d
Merge pull request #2304 from xiexinch/fix_swin_load
MeowZheng Nov 15, 2022
8fed7f5
add data_preprocessor ut
xiexinch Nov 17, 2022
fd0fffc
Update visualization.md
matrixgame2018 Nov 18, 2022
2d4b113
[Fix] Fix typo of BaseSegDataset docstring
MengzhangLI Nov 18, 2022
2f15bfe
Merge pull request #2322 from MengzhangLI/fix_basesegdataset_typo
MeowZheng Nov 19, 2022
f915df4
remove gt padding
xiexinch Nov 19, 2022
4e12bcd
fix data sample is None
xiexinch Nov 19, 2022
c56a299
Merge pull request #2290 from xiexinch/fix_gt_padding
MeowZheng Nov 19, 2022
bc013fb
fixed the AttributeError in show_result_pyplot()
VoyagerXvoyagerx Nov 19, 2022
3bf399e
Add PyTorch installation before import torch.
VoyagerXvoyagerx Nov 21, 2022
5ef9f62
Update demo/MMSegmentation_Tutorial.ipynb
MeowZheng Nov 21, 2022
2b4dbcf
Update demo/MMSegmentation_Tutorial.ipynb
MeowZheng Nov 21, 2022
ca1dc71
Update demo/MMSegmentation_Tutorial.ipynb
MeowZheng Nov 21, 2022
8ea777e
Merge pull request #2326 from VoyagerXvoyagerx/dev-1.x
MeowZheng Nov 21, 2022
cfab5b3
[Fix] Fix ignore class id from -1 to 255 in BaseSegDataset in 1.x
MengzhangLI Nov 21, 2022
6d7af07
Merge pull request #2332 from MengzhangLI/fix_basesegdataset_ignoreid
MeowZheng Nov 22, 2022
58e73c3
[Fix] Fix KNet IterativeDecodeHead bug in dev-1.x branch
MengzhangLI Nov 22, 2022
056edc8
init
xiexinch Nov 22, 2022
8b61820
add comment
MengzhangLI Nov 22, 2022
25abd9e
delete data link
MengzhangLI Nov 22, 2022
14a1bb6
[Fix] Add data string in .gitignore file
MengzhangLI Nov 22, 2022
b19e54c
Merge pull request #2334 from MengzhangLI/fix_knet_dev1.x
MeowZheng Nov 22, 2022
d62774c
Merge pull request #2336 from MengzhangLI/fix_gitignore_dev1.x
MeowZheng Nov 22, 2022
840b5b7
Update visualization.md
matrixgame2018 Nov 23, 2022
721f4d0
add ut
xiexinch Nov 23, 2022
68e3b25
use mmcv.RandomChoiceResize as entry
xiexinch Nov 24, 2022
1910dff
minor change
xiexinch Nov 24, 2022
721afae
add docstring
xiexinch Nov 24, 2022
5d3ecd1
Update visualization.md
matrixgame2018 Nov 29, 2022
ccc2005
Update visualization.md
matrixgame2018 Nov 29, 2022
6d3464a
fix lint
xiexinch Nov 29, 2022
aefcab3
Merge pull request #2298 from matrixgame2018/patch-1
MeowZheng Nov 29, 2022
e98231b
[Enhance] Make scipy as a default dependency in runtime 1.x
MengzhangLI Nov 30, 2022
9251100
[Enhance] Make scipy as a default dependency in runtime in dev-1.x
MengzhangLI Nov 30, 2022
383826f
Merge pull request #2362 from MengzhangLI/scipy_1.x
MeowZheng Nov 30, 2022
5a62b03
fix h,w order
xiexinch Dec 1, 2022
3b731ed
update copyright
xiexinch Dec 1, 2022
0cdab72
Merge pull request #2339 from xiexinch/resize-shortest-edge
MeowZheng Dec 1, 2022
90c816b
[Enchancement] Delete mmcls in runtime.txt in dev-1.x (#2368)
MengzhangLI Dec 1, 2022
925faea
[Fix] Fix the problem of post-processing not removing padding (#2367)
xiexinch Dec 1, 2022
933e4d3
[Feature] Support MaskFormer(NeurIPS'2021) in MMSeg 1.x (#2215)
MengzhangLI Dec 1, 2022
163277b
[Feature] Support Mask2former in MMSeg 1.x (#2255)
xiexinch Dec 5, 2022
f3db8d7
[Fix]Add input argument for datasets (#2379)
MeowZheng Dec 5, 2022
3bac6ef
Fix typo in warning on binary classification (#2382)
nulam Dec 6, 2022
750bb4f
Bump 1.0.0rc2 (#2384)
MeowZheng Dec 6, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
[Doc] Updata transforms Doc
  • Loading branch information
MengzhangLI committed Sep 19, 2022
commit 89f6647d1f6b62d9f02f966e36a7329b2c71d127
125 changes: 104 additions & 21 deletions docs/en/advanced_guides/transforms.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# Data Transforms

In this tutorial, we introduce the design of transforms pipeline in MMSegmentation.

The structure of this guide is as follows:

- [Data Transforms](#data-transforms)
- [Design of Data pipelines](#design-of-data-pipelines)
- [Customization data transformation](#customization-data-transformation)

## Design of Data pipelines

Following typical conventions, we use `Dataset` and `DataLoader` for data loading
Expand All @@ -10,6 +18,24 @@ we introduce a new `DataContainer` type in MMCV to help collect and distribute
data of different size.
See [here](https://github.com/open-mmlab/mmcv/blob/master/mmcv/parallel/data_container.py) for more details.

In 1.x version of MMSegmentation, all data transformations are inherited from `BaseTransform`.
The input and output types of transformations are both dict. A simple example is as follow:

```python
>>> from mmseg.datasets.transforms import LoadAnnotations
>>> transforms = LoadAnnotations()
>>> img_path = './data/cityscapes/leftImg8bit/train/aachen/aachen_000000_000019_leftImg8bit.png.png'
>>> gt_path = './data/cityscapes/gtFine/train/aachen/aachen_000015_000019_gtFine_instanceTrainIds.png'
>>> results = dict(
>>> img_path=img_path,
>>> seg_map_path=gt_path,
>>> reduce_zero_label=False,
>>> seg_fields=[])
>>> data_dict = transforms(results)
>>> print(data_dict.keys())
dict_keys(['img_path', 'seg_map_path', 'reduce_zero_label', 'seg_fields', 'gt_seg_map'])
```

The data preparation pipeline and the dataset is decomposed. Usually a dataset
defines how to process the annotations and a data pipeline defines all the steps to prepare a data dict.
A pipeline consists of a sequence of operations. Each operation takes a dict as input and also output a dict for the next transform.
Expand Down Expand Up @@ -43,47 +69,104 @@ test_pipeline = [
]
```

For each operation, we list the related dict fields that are added/updated/removed.
Before pipelines, the information we can directly obtain from the datasets are img_path, seg_map_path.
For each operation, we list the related dict fields that are `added`/`updated`/`removed`.
Before pipelines, the information we can directly obtain from the datasets are `img_path` and `seg_map_path`.

### Data loading

`LoadImageFromFile`
`LoadImageFromFile`: Load an image from file.

- add: img, img_shape, ori_shape
- add: `img`, `img_shape`, `ori_shape`

`LoadAnnotations`
`LoadAnnotations`: Load semantic segmentation maps provided by dataset.

- add: seg_fields, gt_seg_map
- add: `seg_fields`, `gt_seg_map`

### Pre-processing

`RandomResize`
`RandomResize`: Random resize image & segmentation map.

- add: scale, scale_factor, keep_ratio
- update: img, img_shape, gt_seg_map
- add: `scale`, `scale_factor`, `keep_ratio`
- update: `img`, `img_shape`, `gt_seg_map`

`Resize`
`Resize`: Resize image & segmentation map.

- add: scale, scale_factor, keep_ratio
- update: img, gt_seg_map, img_shape
- add: `scale`, `scale_factor`, `keep_ratio`
- update: `img`, `gt_seg_map`, `img_shape`

`RandomCrop`
`RandomCrop`: Random crop image & segmentation map.

- update: img, pad_shape, gt_seg_map
- update: `img`, `gt_seg_map`, `img_shape`.

`RandomFlip`
`RandomFlip`: Flip the image & segmentation map.

- add: flip, flip_direction
- update: img, gt_seg_map
- add: `flip`, `flip_direction`
- update: `img`, `gt_seg_map`

`PhotoMetricDistortion`
`PhotoMetricDistortion`: Apply photometric distortion to image sequentially,
every transformation is applied with a probability of 0.5.
The position of random contrast is in second or second to last(mode 0 or 1 below, respectively).

- update: img
```
1. random brightness
2. random contrast (mode 0)
3. convert color from BGR to HSV
4. random saturation
5. random hue
6. convert color from HSV to BGR
7. random contrast (mode 1)
```

- update: `img`

### Formatting

`PackSegInputs`
`PackSegInputs`: Pack the inputs data for the semantic segmentation.

- add: inputs, data_sample
- add: `inputs`, `data_sample`
- remove: keys specified by `meta_keys` (merged into the metainfo of data_sample), all other keys

## Customization data transformation

The customized data transformation must inherinted from `BaseTransform` and implement `transform` function.
Here we use a simple flipping transformation as example:

```python
import random
import mmcv
from mmcv.transforms import BaseTransform, TRANSFORMS

@TRANSFORMS.register_module()
class MyFlip(BaseTransform):
def __init__(self, direction: str):
super().__init__()
self.direction = direction

def transform(self, results: dict) -> dict:
img = results['img']
results['img'] = mmcv.imflip(img, direction=self.direction)
return results
```

Thus, we can instantiate a `MyFlip` object and use it to process the data dict.

```python
import numpy as np

transform = MyFlip(direction='horizontal')
data_dict = {'img': np.random.rand(224, 224, 3)}
data_dict = transform(data_dict)
processed_img = data_dict['img']
```

Or, we can use `MyFlip` transformation in data pipeline in our config file.

```python
pipeline = [
...
dict(type='MyFlip', direction='horizontal'),
...
]
```

Note that if you want to use `MyFlip` in config, you must ensure the file containing `MyFlip` is imported during the program run.