Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support missing values in dot #42812

Closed
wants to merge 1 commit into from
Closed

Support missing values in dot #42812

wants to merge 1 commit into from

Conversation

nalimilan
Copy link
Member

@nalimilan nalimilan commented Oct 26, 2021

This is more consistent with matrix product, which propagates missing automatically since * and + are defined for it.

This is needed to avoid a regression in Statistics.cov in Julia 1.7 compared with 1.6. https://github.com/JuliaLang/Statistics.jl/pull/85 replaced a call to sum(conj(y[i])*x[i] for i in eachindex(y, x)) with dot(y, x), which is numerically more stable and was originally used before Statistics was moved to the stdlib (at the time LinearAlgebra was still in Base so dot could not be used from an stdlib). But this change currently makes cov throw an error when the input contains missing, which was not the case in Julia 1.6. This currently makes StatsBase tests fail on Julia 1.7.

EDIT: a solution in Statistics is to use adjoint(y) * x instead of dot(y, x), as the former falls back to the latter for arrays of Numbers but to sum(uu*vv for (uu, vv) in zip(u, v)) for other types. See https://github.com/JuliaLang/Statistics.jl/pull/94. Though it can still be useful to support missing values with dot.

This is more consistent with matrix product, which propagates `missing`
automatically since `*` and `+` are defined for it.
@nalimilan nalimilan added domain:linear algebra Linear algebra domain:missing data Base.missing and related functionality labels Oct 26, 2021
@dkarrasch
Copy link
Member

x-ref #40769 🤣

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
domain:linear algebra Linear algebra domain:missing data Base.missing and related functionality
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants