transform_video_subject()

[laynr]

Idea:

It would be neat to have a transform_video_subject() function. Similar to plot_subject(), where I can provide a torchio.Subject, but instead of just plotting/displaying the middle slice it generates every slice and creates a video transfor as show in https://torchio.readthedocs.io/auto_examples/plot_video.html#sphx-glr-download-auto-examples-plot-video-py.

Background:

My current code to generate a dataset looks like this:

def data_preparation(patients):
    subjects  = []
    labels_df = pd.read_csv(data_dir / 'train_labels.csv', index_col=0)
    # loop thru patients
    for patient in patients:
        # get label for patient
        label = labels_df._get_value(int(patient), 'MGMT_value')
        # create subject object for each patient
        subject = tio.Subject(
            BraTS21ID=patient,
            MGMT_value=label,
            FLAIR=tio.ScalarImage(training_dir / patient / 'FLAIR',),
            T1w=tio.ScalarImage(training_dir / patient / 'T1w',),
            T1wCE=tio.ScalarImage(training_dir / patient / 'T1wCE',),
            T2w=tio.ScalarImage(training_dir / patient / 'T2w',),
         )
        # add subject object to subjects list
        subjects.append(subject)

    # preprocessing transforms
    preprocessing_transforms = tio.Compose([
        tio.ToCanonical(),
        tio.Resample(1, image_interpolation='bspline'),
        tio.Resample('T1w', image_interpolation='nearest'),
        tio.RescaleIntensity((-1, 1)),
        tio.CropOrPad((280, 280, 264)),
        tio.OneHot(),
    ])
        

    # create datasets from transformed subjects
    dataset = tio.SubjectsDataset(subjects, transform=preprocessing_transforms)
    print(f'patients :{len(dataset)}')

    
    return dataset

# create training and validation datasets    
dataset = data_preparation(patients) 

I then iterate thru the dataset, plot, and save the subjects:

    for subject in tqdm(iterable):
        filename = out_dir / f'{subject["BraTS21ID"]}_{subject["MGMT_value"]}.png'
        subject.plot(reorient=False, output_path=filename)

Full Kaggle notebook available at: https://www.kaggle.com/ohbewise/s4p1-collage
This creates awesome synchronized plots of the middle slice like this:

Today I am going to be working in matplotlib.pyplot to save every slice of the subject and create 12 synchronized side-by-side plot videos similar to this video of one image:

video.mp4

I feel it would be beneficial for others so I am creating a new discussion here.

Other

In my use case I will be also removing all plot titles, subtitles, and x/y scaling - smashing all 12 images together by cropping out the extra pixels, so they can be used in 2D machine learning.

[laynr]

Will also be discussed at https://www.kaggle.com/c/rsna-miccai-brain-tumor-radiogenomic-classification/discussion/276371

[fepegar]

I think this would be very cool. It could be a new method plot_animation, for example. Thanks for opening the discussion!

Have you also seen the new to_gif method of Image? It might offer some tips for the implementation. Maybe the code could be even reused.

P.S.: good luck with the challenge! I think it's tougher than it looks.

[laynr]

@fepegar does this look right? I don't know MRI's well enough to know if it is a synchronized animations

I not going for most efficient code, but I am going for synchronized MRI accuracy.
I basically just modified plot_volume to take indices as a argument and then called it 'depth of volume' times. Something like below:

for x in range(<depth of volume>):
    subject.plot(indices= (x,x,x), output_path=filename, show=False) 

visualization.py

    if indices is None:
        indices = np.array(data.shape) // 2
    i, j, k = indices
    slice_x = rotate(data[i, :, :], radiological=radiological)
    slice_y = rotate(data[:, j, :], radiological=radiological)
    slice_z = rotate(data[:, :, k], radiological=radiological)

Then used imageio to make gif. Notebook is forthcoming.

[laynr]

• Full resolution can be see here the one above had to be resized to comply with gitlab's 10mb upload limit.
• This is the notebook used to create gif.
• Pull request summitted, do as you want with it.
• Updated pull request from branch summitted, do as you want with it.