Different image_interpolation arguments for each element of a dictionary input

[stalhabukhari]

Hi!

I'm new to TorchIO so forgive me if I miss any obvious and already clarified point. I wish to use the torchio.transforms module, and I am currently planning to do something like below:

input_dict = {'data': data_volume, 'label': label_volume}
fn = torchio.transforms.RandomAffine(include=['data', 'label'], image_interpolation='linear', **args)
output_dict = fn(input_dict)

I wish to use a different interpolation method for different inputs (e.g., 'linear' for 'data' and 'nearest' for 'label'). After a couple of clicks, I found that the only way to do so is to create a torchio.Subject instance and specify the label there. I wish to bypass this need and work directly with dictionaries. Is it possible to do so? Something like image_interpolation = {'data': 'linear', 'label': 'nearest'} would be convenient.

[fepegar]

Hi, @stalhabukhari. It's a legit question and you're right, that's the only way for now.

If there is interest, we can add support to pass a dictionary of interpolations. That will also improve compatibility with related libraries. I could work on this next month.

Ouf of curiosity, why don't you use Subject? Note that it is a subclass of dict:

In [1]: import torchio as tio

In [2]: colin = tio.datasets.Colin27()

In [3]: isinstance(colin, dict)
Out[3]: True

[stalhabukhari]

If there is interest, we can add support to pass a dictionary of interpolations. That will also improve compatibility with related libraries. I could work on this next month.

Thank you so much!

Ouf of curiosity, why don't you use Subject? Note that it is a subclass of dict:

Oh right! I didn't know that. My use-case was to perform data augmentation on-the-fly, and I was trying to guess what type of inputs I should pass to the transform so that the same transformation was applied to MRI segment as well as Label map, but with different interpolation methods. I read that include and exclude arguments are a way of deciding which inputs should be subjected to a transform (e.g., Label maps should be excluded from noise distortions). I was thinking along the same lines and so I tried to search in the source code if the interpolation method could be selected by providing correspondence to input dictionary elements.

So, if I did something like:

# Method 1
input_dict = {'data': torchio.ScalarImage(tensor=data_volume), 'label': torchio.LabelMap(tensor=label_volume)}
fn = torchio.transforms.RandomAffine(include=['data', 'label'], image_interpolation='linear')
output_dict = fn(input_dict)

# Method 2
input_subj = torchio.Subject({'data': torchio.ScalarImage(tensor=data_volume), 'label': torchio.LabelMap(tensor=label_volume)})
fn = torchio.transforms.RandomAffine(include=['data', 'label'], image_interpolation='linear')
output_subj = fn(input_subj)

Are both the methods equivalent?

[fepegar]

Ok, I think I was right to suspect that you don't need a dict at all. If you run:

input_subj = torchio.Subject({'data': torchio.ScalarImage(tensor=data_volume), 'label': torchio.LabelMap(tensor=label_volume)})
# Alternative syntax
input_subj = torchio.Subject(data=torchio.ScalarImage(tensor=data_volume), label=torchio.LabelMap(tensor=label_volume))
fn = torchio.transforms.RandomAffine()
output_subj = fn(input_subj)

you will have applied the same transform to both images. The resampling is nearest neighbor for label maps, and linear by default for everything else. And the transform is applied to all images by default, so you don't need include at all.
Basically, you don't need to specify anything. All the defaults do what you want :)

In [1]: import torchio as tio

In [2]: colin = tio.datasets.Colin27()

In [3]: colin.t1
Out[3]: ScalarImage(path: "/home/fernando/.cache/torchio/mni_colin27_1998_nifti/colin27_t1_tal_lin.nii")

In [5]: colin.brain
Out[5]: LabelMap(path: "/home/fernando/.cache/torchio/mni_colin27_1998_nifti/colin27_t1_tal_lin_mask.nii")

In [6]: colin.plot()

In [7]: transform = tio.RandomAffine()

In [8]: transformed = transform(colin)

In [9]: transformed.plot()

[stalhabukhari]

Awesome!
After having to revisit my data augmentation pipeline every 3 to 4 months, I think I'll stick with this for longer. Thank you for the discussion!

[fepegar]

No worries!