Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LNCC Training: Unable to learn (very unstable) #527

Open
JannikLa opened this issue Jun 7, 2023 · 1 comment
Open

LNCC Training: Unable to learn (very unstable) #527

JannikLa opened this issue Jun 7, 2023 · 1 comment

Comments

@JannikLa
Copy link

JannikLa commented Jun 7, 2023

Task (what are you trying to do/register?)

I am registering two T1 3D images from the OASIS data set (L2Reg challenge).
The scans are preprocessed (skull-stripped, aligned, normalized).
I am taking a fixed image from one subject and randomly select another subject as moving image (examples below).

I invested a lot if time in figuring out what is wrong. Maybe someone can give hints on what I could try or what might be wrong in my setup. Any help is appreciated!

What have you tried

I am using the pytroch implemention of voxelmorph. A dataloader loads the inputs to the network.

  • The network works as expected with MSE
  • With LNCC the loss is very unstable an does not decrease

Details of experiments

  • My "basline" setup is: lr: 1e-4, bs: 4, Reg weight: 0.1, loss: LNCC (win size 9)
    • I changed hyper-parameters separately in both directions (e.g. increasing and decreasing the learning rate)
    • I also tried setting the Regularization weight to zero
    • If I set the initialization of the flow layer a lot higher (to 1 or 10) the network is able to overfit
      • self.flow.weight = nn.Parameter(Normal(0, 1e-5).sample(self.flow.weight.shape)) (networks.py, l 214)
  • I also tried debugging the LNCC loss. It works as expected (e.g. tried computing the loss between identical images)

Image of training loss with maxed out smoothing in visualization ("baseline" setup from above)

image

Image of training loss (LNCC) when overfitting on a pair of images with large initialization weights (Normal(0, 10))

image

Image of input data

image

image

@JannikLa JannikLa changed the title LNNC Training: Unable to learn (very unstable) LNCC Training: Unable to learn (very unstable) Jun 7, 2023
@adalca
Copy link
Collaborator

adalca commented Jun 12, 2023

I haven't worked with the pytorch version that much especially recently, but just so I know -- when you say the MSE version worked, what were the hyperparameters that you used?

What do you mean by this line -- what does 'overfit' mean here?

If I set the initialization of the flow layer a lot higher (to 1 or 10) the network is able to overfit

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants