Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port changes from lite branch #246

Open
pmelchior opened this issue Jul 2, 2021 · 1 comment
Open

Port changes from lite branch #246

pmelchior opened this issue Jul 2, 2021 · 1 comment

Comments

@pmelchior
Copy link
Owner

There are three distinct changes in the lite brach that could be brought into the main branch, depending on further testing

  • a custom grad function for the FFT-based convolution (in ConvolutionRenderer) that avoids the autograd multi-step gradient in favor of the convolution with the transpose of the diff kernel.
  • reduction of the step size of the first fit iteration to 1/10 of its nominal value to prevent strong jumps for a well-initialized model at the very beginning when amsgrad has no previous gradient information
  • and effective L1 update that is expressed at the observation level, not the parameter level. This solves the problem of finding an appropriate threshold but is formally problematic because it's not a real prox.
@fred3m
Copy link
Collaborator

fred3m commented Jul 2, 2021

Thanks for opening this ticket, sorry that I didn't get around to it yet. You might also want to consider porting the implementation of parameters in proxmin as opposed to functions that fit them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants