-
Notifications
You must be signed in to change notification settings - Fork 83
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
The pyglmnet solvers give different answers and neither solver agrees with sklearn #395
Comments
Do you mind checking on the development version? I get: pyglmnet gradient vs cd 0.8964272943835351 pyglmnet gradient vs cd norm diff 1.390974023209033 we have made some fixes since the release and I'm pretty confident the batch-gradient solver is correct. Regarding |
I installed version 1.2.dev0 from github get the same answers as you do above i.e. batch-gradient seems correct but there seems to be an issue with cdfast. |
I am also currently experiencing this issue and investigating |
The pyglment package gives different estimated coefficients for linear regression depending on which solver you select (e.g. cdfast or batch-gradient). Neither solver agrees with sklearn (I believe they should agree!) The code below gives an example for both ElasticNet and vanilla Lasso.
Reproducible examples
ElasticNet
First let's check elastic net
And we see the solutions are different:
I messed around with the optimization parameters (e.g. tol, learning_rate, max_iter) and could not get anyone to agree. Of course the word "different" is subject to what you think the numerical tolerance should be but, but these norms seem large enough to cause concern.
I should note here that if you do just ridge regression (i.e. l1_ratio=0) the answers do seem to agree.
Lasso
Again we get different answers
Perhaps coordinate descent is getting close to sklearn, but I think that norm is still large enough to be concerning.
Software versions
I am using Python 3.6.12, pyglmnet 1.1 and sklean 0.24.1
The text was updated successfully, but these errors were encountered: