Request: doubts about training process

[agustinbc]

Hello Manuel,

I've been running and using the notebooks you wrote to reproduce results on cMAC and ATLAS. I also read most of the code in the repository.

We have some doubts regarding the training process. Specifically, we would like to know how the training data is organized: the input to the network is arranged as pairs of 4-dimensional matrices (that is, [time0, time0, ...], [time1, time2, ...]) or the inputs are pairs of 3-dimensional matrices (that is, time 0 and time t)?

If there's any chance you could share the training.py script so as to understand better the training process and replicate the results on an end to end fashion, we would really appreciate it.

In any case, I would like to thank you for this work because this is a significant contribution to the field.

Thanks in advance.

Best,

Agustín Bernardo

[moralesq]

Hi Agustin, unfortunately I haven't found the time to integrate the training scripts with this repo, and most likely I will create a new repo in the future with a more optimize implementation.

For some clarification:

The input to the U-Net was a two-channel array of size N×N×Nz×2, which consisted of a pair of short axis cine stacks 〖[V(t=0),V(t≥0)] concatenated along the channel dimension. Each stack V(t) consisted of Nz slices of size N×N covering the LV at a particular time frame. The first channel in the array was set to the end-diastolic (ED) frame at t=0, while the second channel included arbitrary frames at t≥0. Thus, the outputs of the U-Net were the displacement vectors u(t) of size N×N×Nz×3, where each channel corresponds to the displacement from V(t=0) to V(t≥0) along x, y, and z directions.

In other words, for a cine dataset with Nt time frames:

u(t=1) = CarMEN([V(0), V(t=1)])
u(t=2) = CarMEN([V(0), V(t=2)])
.
.
.
u(t=T) = CarMEN([V(0), V(t=T)])

I hope this helps.

[agustinbc]

I do correctly understand that. Nonetheless, in the ACDC notebook two (1, N, N, Nz, 30) shaped images are used for motion estimation inference:

    V_0 = np.repeat(V[:1], len(V)-1, axis=0)[...,None]
    V_t = V[1:][...,None]
    
    y_t = netME([V_0, V_t]).numpy()

and in the cMAC notebook two (1, N, N, Nz, 1) images are used to estimate ED-ES motion:

    V_0 = V[...,0][None,...,None]
    V_t = V[...,ES][None,...,None]

    y_t = netME([V_0, V_t]).numpy()

After your answer, I will make the assumption that the second case -that is, passing pairs of 3D images as a list- was used on training time - is this hypothesis correct?

As for the training script, I understand the intention of iterating the code in this repository onto a more optimized version. Anyhow, having it even in a semi-functional state would be immensely useful to replicate the methodology.

Thanks for being so responsive!

Cheers,

Agustín