Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve numerical stability of cov #85

Merged
merged 2 commits into from
Sep 13, 2021
Merged

Improve numerical stability of cov #85

merged 2 commits into from
Sep 13, 2021

Conversation

nalimilan
Copy link
Member

cov(::AbstractVector, ::AbstractVector) was less stable than other cov methods and than var on arrays as it called sum on a generator, which does not use pairwise summation. Use a Broadcasted object instead to benefit from pairwise summation.

Fixes https://github.com/JuliaLang/Statistics.jl/issues/83.

Performance also seems to increase a bit:

# Before
julia> A = 20*randn(Float32, 10_000_000) .+ 100;

julia> @btime cov(A, A);
  42.300 ms (5 allocations: 76.29 MiB)

# After
julia> @btime cov(A, A);
  33.832 ms (5 allocations: 76.29 MiB)

`cov(::AbstractVector, ::AbstractVector)` was less stable than other `cov`
methods and than `var` on arrays as it called `sum` on a generator, which
does not use pairwise summation. Use a `Broadcasted` object instead
to benefit from pairwise summation.
@codecov
Copy link

codecov bot commented Sep 7, 2021

Codecov Report

Merging #85 (e4a068d) into master (54f9b0d) will decrease coverage by 1.28%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master      #85      +/-   ##
==========================================
- Coverage   98.18%   96.89%   -1.29%     
==========================================
  Files           1        1              
  Lines         386      419      +33     
==========================================
+ Hits          379      406      +27     
- Misses          7       13       +6     
Impacted Files Coverage Δ
src/Statistics.jl 96.89% <100.00%> (-1.29%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 54f9b0d...e4a068d. Read the comment docs.

src/Statistics.jl Outdated Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cov Float32 precision issues
2 participants