Skip to content

Latest commit

 

History

History
314 lines (293 loc) · 18.5 KB

report.md

File metadata and controls

314 lines (293 loc) · 18.5 KB
Raw

Benchmark Report

Job Properties

Commits: gbaraldi/julia@566c5f9e1d3a5e543c0b095ac208aa0803f2f9a1 vs JuliaLang/julia@cae7fefdc0d381a0615b50990f90c86def813662

Comparison Diff: link

Triggered By: link

Tag Predicate: !"scalar"

Results

Note: If Chrome is your browser, I strongly recommend installing the Wide GitHub extension, which makes the result table easier to read.

Below is a table of this job's results, obtained by running the benchmarks found in JuliaCI/BaseBenchmarks.jl. The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.

The percentages accompanying time and memory values in the below table are noise tolerances. The "true" time/memory value for a given benchmark is expected to fall within this percentage of the reported value.

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results that indicate possible regressions or improvements - are shown below (thus, an empty table means that all benchmark results remained invariant between builds).

ID time ratio memory ratio
["array", "cat", "4467"] 1.06 (5%) ❌ 1.00 (1%)
["array", "cat", ("catnd", 5)] 1.10 (5%) ❌ 1.00 (1%)
["array", "comprehension", ("collect", "StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}")] 1.77 (5%) ❌ 1.00 (1%)
["array", "comprehension", ("collect", "Vector{Float64}")] 4.08 (5%) ❌ 1.00 (1%)
["array", "comprehension", ("comprehension_collect", "StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}")] 1.77 (5%) ❌ 1.00 (1%)
["array", "comprehension", ("comprehension_collect", "Vector{Float64}")] 2.27 (5%) ❌ 1.00 (1%)
["array", "comprehension", ("comprehension_indexing", "Vector{Float64}")] 1.35 (30%) ❌ 1.00 (1%)
["array", "comprehension", ("comprehension_iteration", "StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}")] 1.22 (5%) ❌ 1.00 (1%)
["array", "comprehension", ("comprehension_iteration", "Vector{Float64}")] 1.38 (5%) ❌ 1.00 (1%)
["array", "equality", ("==", "BitArray")] 0.92 (5%) ✅ 1.00 (1%)
["array", "equality", ("==", "Vector{Int64}")] 1.16 (5%) ❌ 1.00 (1%)
["array", "index", ("sumrange", "SubArray{Float32, 2, Base.ReshapedArray{Float32, 2, SubArray{Float32, 3, Array{Float32, 3}, Tuple{Base.Slice{Base.OneTo{Int64}}, Base.Slice{Base.OneTo{Int64}}, Base.Slice{Base.OneTo{Int64}}}, true}, Tuple{}}, Tuple{Base.Slice{Base.OneTo{Int64}}, UnitRange{Int64}}, true}")] 0.40 (50%) ✅ 1.00 (1%)
["array", "reductions", ("sumabs2", "Float64")] 0.92 (5%) ✅ 1.00 (1%)
["array", "reverse", "rev_load_fast!"] 1.06 (5%) ❌ 1.00 (1%)
["array", "setindex!", ("setindex!", 3)] 0.93 (5%) ✅ 1.00 (1%)
["broadcast", "typeargs", ("tuple", 10)] 0.92 (5%) ✅ 1.00 (1%)
["dates", "parse", "Date"] 1.07 (5%) ❌ 1.00 (1%)
["dates", "parse", "DateTime"] 0.94 (5%) ✅ 1.00 (1%)
["find", "findall", ("> q0.5", "Vector{Int8}")] 1.05 (5%) ❌ 1.00 (1%)
["find", "findall", ("> q0.8", "Vector{Float64}")] 1.18 (5%) ❌ 1.00 (1%)
["find", "findall", ("> q0.8", "Vector{Int8}")] 1.14 (5%) ❌ 1.00 (1%)
["find", "findall", ("> q0.95", "Vector{Int8}")] 1.08 (5%) ❌ 1.00 (1%)
["find", "findall", ("> q0.99", "Vector{Float64}")] 1.05 (5%) ❌ 1.00 (1%)
["find", "findall", ("> q0.99", "Vector{Int8}")] 1.06 (5%) ❌ 1.00 (1%)
["find", "findall", ("BitVector", "50-50")] 0.88 (5%) ✅ 1.00 (1%)
["find", "findall", ("BitVector", "90-10")] 0.95 (5%) ✅ 1.00 (1%)
["find", "findall", ("Vector{Bool}", "50-50")] 0.95 (5%) ✅ 1.00 (1%)
["find", "findall", ("ispos", "Vector{Float32}")] 0.94 (5%) ✅ 1.00 (1%)
["find", "findall", ("ispos", "Vector{Float64}")] 0.94 (5%) ✅ 1.00 (1%)
["find", "findall", ("ispos", "Vector{UInt8}")] 0.94 (5%) ✅ 1.00 (1%)
["find", "findnext", ("ispos", "Vector{Bool}")] 1.06 (5%) ❌ 1.00 (1%)
["find", "findprev", ("BitVector", "10-90")] 1.05 (5%) ❌ 1.00 (1%)
["find", "findprev", ("ispos", "Vector{Float32}")] 1.13 (5%) ❌ 1.00 (1%)
["find", "findprev", ("ispos", "Vector{UInt8}")] 1.05 (5%) ❌ 1.00 (1%)
["inference", "allinference", "quadratic"] 1.05 (5%) ❌ 1.00 (1%)
["inference", "optimization", "domsort_ssa!"] 1.08 (5%) ❌ 1.00 (1%)
["inference", "optimization", "quadratic"] 1.12 (5%) ❌ 1.00 (1%)
["inference", "optimization", "sin(42)"] 1.08 (5%) ❌ 1.00 (1%)
["io", "serialization", ("deserialize", "Matrix{Float64}")] 0.92 (5%) ✅ 1.00 (1%)
["io", "serialization", ("serialize", "Matrix{Float64}")] 1.12 (5%) ❌ 1.00 (1%)
["linalg", "arithmetic", ("+", "Vector", "Vector", 256)] 0.94 (45%) 0.99 (1%) ✅
["linalg", "arithmetic", ("+", "typename(Bidiagonal)", "typename(Bidiagonal)", 256)] 1.01 (45%) 0.99 (1%) ✅
["linalg", "arithmetic", ("+", "typename(Diagonal)", "typename(Diagonal)", 256)] 0.93 (45%) 0.99 (1%) ✅
["linalg", "arithmetic", ("+", "typename(SymTridiagonal)", "typename(SymTridiagonal)", 256)] 1.09 (45%) 0.99 (1%) ✅
["linalg", "arithmetic", ("+", "typename(Tridiagonal)", "typename(Tridiagonal)", 256)] 0.98 (45%) 0.99 (1%) ✅
["linalg", "arithmetic", ("-", "Vector", "Vector", 256)] 0.94 (45%) 0.99 (1%) ✅
["linalg", "arithmetic", ("-", "typename(Bidiagonal)", "typename(Bidiagonal)", 256)] 1.16 (45%) 0.99 (1%) ✅
["linalg", "arithmetic", ("-", "typename(Diagonal)", "typename(Diagonal)", 256)] 0.87 (45%) 0.99 (1%) ✅
["linalg", "arithmetic", ("-", "typename(SymTridiagonal)", "typename(SymTridiagonal)", 256)] 1.05 (45%) 0.99 (1%) ✅
["linalg", "arithmetic", ("-", "typename(Tridiagonal)", "typename(Tridiagonal)", 256)] 1.06 (45%) 0.99 (1%) ✅
["micro", "fib"] 1.14 (5%) ❌ 1.00 (1%)
["misc", "23042", "ComplexF32"] 1.13 (5%) ❌ 1.00 (1%)
["misc", "23042", "Float64"] 1.13 (5%) ❌ 1.00 (1%)
["misc", "afoldl", "Int"] 1.08 (5%) ❌ 1.00 (1%)
["misc", "bitshift", ("Int", "Int")] 0.79 (5%) ✅ 1.00 (1%)
["misc", "bitshift", ("Int", "UInt")] 0.85 (5%) ✅ 1.00 (1%)
["misc", "bitshift", ("UInt", "UInt")] 0.85 (5%) ✅ 1.00 (1%)
["misc", "bitshift", ("UInt32", "UInt32")] 0.91 (5%) ✅ 1.00 (1%)
["misc", "foldl", "foldl(+, filter(...); init = 0.0)"] 1.06 (5%) ❌ 1.00 (1%)
["misc", "perf highdim generator"] 1.09 (5%) ❌ 1.00 (1%)
["random", "ranges", ("rand", "ImplicitRNG", "Int", "1:1000")] 0.74 (25%) ✅ 1.00 (1%)
["random", "types", ("rand", "ImplicitRNG", "Float64")] 0.68 (25%) ✅ 1.00 (1%)
["random", "types", ("rand", "ImplicitRNG", "ImplicitFloat64")] 0.68 (25%) ✅ 1.00 (1%)
["random", "types", ("rand", "ImplicitRNG", "Int64")] 0.74 (25%) ✅ 1.00 (1%)
["random", "types", ("randn", "ImplicitRNG", "Float64")] 0.72 (25%) ✅ 1.00 (1%)
["random", "types", ("randn", "ImplicitRNG", "ImplicitFloat64")] 0.72 (25%) ✅ 1.00 (1%)
["simd", ("Cartesian", "axpy!", "Int32", 3, 31)] 1.21 (20%) ❌ 1.00 (1%)
["simd", ("CartesianPartition", "conditional_loop!", "Int32", 2, 31)] 1.21 (20%) ❌ 1.00 (1%)
["simd", ("CartesianPartition", "manual_example!", "Float64", 2, 64)] 1.24 (20%) ❌ 1.00 (1%)
["sparse", "constructors", ("Diagonal", 1000)] 0.92 (5%) ✅ 1.00 (1%)
["sparse", "constructors", ("IJV", 10)] 1.12 (5%) ❌ 1.00 (1%)
["sparse", "constructors", ("IV", 10)] 1.18 (5%) ❌ 1.00 (1%)
["sparse", "sparse matvec", "adjoint"] 0.82 (5%) ✅ 1.00 (1%)
["sparse", "sparse matvec", "non-adjoint"] 0.82 (5%) ✅ 1.00 (1%)
["sparse", "transpose", ("adjoint", "(20000, 20000)")] 1.67 (30%) ❌ 1.00 (1%)
["sparse", "transpose", ("transpose", "(600, 600)")] 1.33 (30%) ❌ 1.00 (1%)
["string", "join"] 0.45 (40%) ✅ 1.00 (1%)
["string", "readuntil", "target length 50000"] 1.05 (5%) ❌ 1.00 (1%)
["string", "repeat", "repeat str len 16"] 0.93 (5%) ✅ 1.00 (1%)
["string", "replace"] 1.08 (5%) ❌ 1.00 (1%)
["tuple", "linear algebra", ("matvec", "(16, 16)", "(16,)")] 0.94 (5%) ✅ 1.00 (1%)
["tuple", "linear algebra", ("matvec", "(8, 8)", "(8,)")] 1.09 (5%) ❌ 1.00 (1%)
["tuple", "reduction", ("minimum", "(2, 2)")] 1.17 (5%) ❌ 1.00 (1%)
["tuple", "reduction", ("minimum", "(2,)")] 0.94 (5%) ✅ 1.00 (1%)
["tuple", "reduction", ("sum", "(16,)")] 0.88 (5%) ✅ 1.00 (1%)
["tuple", "reduction", ("sumabs", "(8,)")] 1.06 (5%) ❌ 1.00 (1%)
["union", "array", ("broadcast", "*", "Int8", "(false, true)")] 0.94 (5%) ✅ 1.00 (1%)
["union", "array", ("broadcast", "abs", "Float64", 1)] 0.94 (5%) ✅ 1.00 (1%)
["union", "array", ("broadcast", "abs", "Int8", 1)] 1.26 (5%) ❌ 1.00 (1%)
["union", "array", ("collect", "all", "BigInt", 0)] 0.93 (5%) ✅ 1.00 (1%)
["union", "array", ("collect", "all", "Int8", 1)] 1.10 (5%) ❌ 1.00 (1%)
["union", "array", ("map", "abs", "Float32", 1)] 1.26 (5%) ❌ 1.00 (1%)
["union", "array", ("map", "abs", "Int64", 1)] 0.90 (5%) ✅ 1.00 (1%)
["union", "array", ("map", "abs", "Int8", 1)] 1.16 (5%) ❌ 1.00 (1%)
["union", "array", ("map", "identity", "BigInt", 0)] 0.93 (5%) ✅ 1.00 (1%)
["union", "array", ("map", "identity", "Int8", 1)] 1.10 (5%) ❌ 1.00 (1%)
["union", "array", ("perf_countequals", "Bool")] 0.94 (5%) ✅ 1.00 (1%)
["union", "array", ("perf_sum3", "ComplexF64", 1)] 0.94 (5%) ✅ 1.00 (1%)
["union", "array", ("perf_sum4", "Bool", 0)] 0.95 (5%) ✅ 1.00 (1%)
["union", "array", ("perf_sum4", "ComplexF64", 1)] 0.76 (5%) ✅ 1.00 (1%)
["union", "array", ("skipmissing", "collect", "Float32", 0)] 0.94 (5%) ✅ 1.00 (1%)
["union", "array", ("skipmissing", "perf_sumskipmissing", "Int8", 0)] 1.06 (5%) ❌ 1.00 (1%)
["union", "array", ("skipmissing", "perf_sumskipmissing", "Union{Missing, BigFloat}", 1)] 0.87 (5%) ✅ 0.87 (1%) ✅
["union", "array", ("skipmissing", "perf_sumskipmissing", "Union{Missing, BigInt}", 1)] 0.87 (5%) ✅ 0.80 (1%) ✅
["union", "array", ("skipmissing", "perf_sumskipmissing", "Union{Missing, Bool}", 1)] 0.87 (5%) ✅ 0.00 (1%) ✅
["union", "array", ("skipmissing", "perf_sumskipmissing", "Union{Missing, ComplexF64}", 1)] 0.81 (5%) ✅ 0.00 (1%) ✅
["union", "array", ("skipmissing", "perf_sumskipmissing", "Union{Missing, Float32}", 1)] 1.02 (5%) 0.00 (1%) ✅
["union", "array", ("skipmissing", "perf_sumskipmissing", "Union{Missing, Float64}", 1)] 0.97 (5%) 0.00 (1%) ✅
["union", "array", ("skipmissing", "perf_sumskipmissing", "Union{Missing, Int64}", 1)] 1.03 (5%) 0.00 (1%) ✅
["union", "array", ("skipmissing", "perf_sumskipmissing", "Union{Missing, Int8}", 1)] 0.85 (5%) ✅ 0.00 (1%) ✅
["union", "array", ("skipmissing", "perf_sumskipmissing", "Union{Nothing, ComplexF64}", 0)] 0.90 (5%) ✅ 1.00 (1%)
["union", "array", ("skipmissing", "sum", "Union{Missing, BigFloat}", 1)] 0.94 (5%) ✅ 1.00 (1%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["alloc"]
  • ["array", "accumulate"]
  • ["array", "any/all"]
  • ["array", "bool"]
  • ["array", "cat"]
  • ["array", "comprehension"]
  • ["array", "convert"]
  • ["array", "equality"]
  • ["array", "growth"]
  • ["array", "index"]
  • ["array", "reductions"]
  • ["array", "reverse"]
  • ["array", "setindex!"]
  • ["array", "subarray"]
  • ["broadcast"]
  • ["broadcast", "dotop"]
  • ["broadcast", "fusion"]
  • ["broadcast", "mix_scalar_tuple"]
  • ["broadcast", "sparse"]
  • ["broadcast", "typeargs"]
  • ["collection", "deletion"]
  • ["collection", "initialization"]
  • ["collection", "iteration"]
  • ["collection", "optimizations"]
  • ["collection", "queries & updates"]
  • ["collection", "set operations"]
  • ["dates", "accessor"]
  • ["dates", "arithmetic"]
  • ["dates", "construction"]
  • ["dates", "conversion"]
  • ["dates", "parse"]
  • ["dates", "query"]
  • ["dates", "string"]
  • ["find", "findall"]
  • ["find", "findnext"]
  • ["find", "findprev"]
  • ["frontend"]
  • ["inference", "abstract interpretation"]
  • ["inference", "allinference"]
  • ["inference", "optimization"]
  • ["io", "array_limit"]
  • ["io", "read"]
  • ["io", "serialization"]
  • ["io"]
  • ["linalg", "arithmetic"]
  • ["linalg", "blas"]
  • ["linalg", "factorization"]
  • ["linalg"]
  • ["micro"]
  • ["misc"]
  • ["misc", "23042"]
  • ["misc", "afoldl"]
  • ["misc", "allocation elision view"]
  • ["misc", "bitshift"]
  • ["misc", "foldl"]
  • ["misc", "issue 12165"]
  • ["misc", "iterators"]
  • ["misc", "julia"]
  • ["misc", "parse"]
  • ["misc", "repeat"]
  • ["misc", "splatting"]
  • ["problem", "chaosgame"]
  • ["problem", "fem"]
  • ["problem", "go"]
  • ["problem", "grigoriadis khachiyan"]
  • ["problem", "imdb"]
  • ["problem", "json"]
  • ["problem", "laplacian"]
  • ["problem", "monte carlo"]
  • ["problem", "raytrace"]
  • ["problem", "seismic"]
  • ["problem", "simplex"]
  • ["problem", "spellcheck"]
  • ["problem", "stockcorr"]
  • ["problem", "ziggurat"]
  • ["random", "collections"]
  • ["random", "randstring"]
  • ["random", "ranges"]
  • ["random", "sequences"]
  • ["random", "types"]
  • ["shootout"]
  • ["simd"]
  • ["sort", "insertionsort"]
  • ["sort", "issorted"]
  • ["sort", "mergesort"]
  • ["sort", "quicksort"]
  • ["sparse", "arithmetic"]
  • ["sparse", "constructors"]
  • ["sparse", "index"]
  • ["sparse", "matmul"]
  • ["sparse", "sparse matvec"]
  • ["sparse", "sparse solves"]
  • ["sparse", "transpose"]
  • ["string", "==(::AbstractString, ::AbstractString)"]
  • ["string", "==(::SubString, ::String)"]
  • ["string", "findfirst"]
  • ["string"]
  • ["string", "readuntil"]
  • ["string", "repeat"]
  • ["tuple", "index"]
  • ["tuple", "linear algebra"]
  • ["tuple", "misc"]
  • ["tuple", "reduction"]
  • ["union", "array"]

Version Info

Primary Build

Julia Version 1.9.0-DEV.942
Commit 566c5f9e1d (2022-07-07 20:17 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 20.04.4 LTS
  uname: Linux 5.4.0-113-generic #127-Ubuntu SMP Wed May 18 14:30:56 UTC 2022 x86_64 x86_64
  CPU: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz: 
              speed         user         nice          sys         idle          irq
       #1  3004 MHz     113895 s        249 s      31650 s   38153909 s          0 s
       #2  3091 MHz    3754212 s         60 s     173248 s   34395064 s          0 s
       #3  2917 MHz      93902 s         28 s      17354 s   38188731 s          0 s
       #4  2885 MHz      68064 s        121 s      17004 s   38206293 s          0 s
       #5  3184 MHz     100600 s         33 s      17297 s   38073981 s          0 s
       #6  2674 MHz      80670 s        124 s      16895 s   38216499 s          0 s
       #7  3223 MHz      98825 s         77 s      17439 s   38204389 s          0 s
       #8  3227 MHz      95134 s         72 s      16792 s   38195534 s          0 s
  Memory: 31.32082748413086 GB (15744.0546875 MB free)
  Uptime: 3.83480213e6 sec
  Load Avg:  1.0  1.0  1.0
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-14.0.5 (ORCJIT, haswell)
  Threads: 1 on 8 virtual cores

Comparison Build

Julia Version 1.9.0-DEV.925
Commit cae7fefdc0 (2022-07-07 20:16 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 20.04.4 LTS
  uname: Linux 5.4.0-113-generic #127-Ubuntu SMP Wed May 18 14:30:56 UTC 2022 x86_64 x86_64
  CPU: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz: 
              speed         user         nice          sys         idle          irq
       #1  3479 MHz     114175 s        249 s      31809 s   38255972 s          0 s
       #2  2906 MHz    3850648 s         60 s     177374 s   34397174 s          0 s
       #3  3461 MHz      93990 s         28 s      17364 s   38291276 s          0 s
       #4  3422 MHz      68119 s        121 s      17015 s   38308810 s          0 s
       #5  3415 MHz     100985 s         33 s      17316 s   38176112 s          0 s
       #6  2696 MHz      81126 s        124 s      16920 s   38318688 s          0 s
       #7  3364 MHz      99418 s         77 s      17465 s   38306437 s          0 s
       #8  3206 MHz      95683 s         72 s      16816 s   38297625 s          0 s
  Memory: 31.32082748413086 GB (15756.6484375 MB free)
  Uptime: 3.84506952e6 sec
  Load Avg:  1.0  1.0  1.0
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-14.0.5 (ORCJIT, haswell)
  Threads: 1 on 8 virtual cores