Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

lambda 3% slower in 1.6 pre-release vs 1.5.3 #38911

Closed
clarkevans opened this issue Dec 16, 2020 · 5 comments
Closed

lambda 3% slower in 1.6 pre-release vs 1.5.3 #38911

clarkevans opened this issue Dec 16, 2020 · 5 comments
Labels
performance Must go faster

Comments

@clarkevans
Copy link
Member

clarkevans commented Dec 16, 2020 •
edited
Loading

For much of the code I'm working with Julia 1.6 seems a good 10-20% faster. For a HypertextLiteral.jl benchmark, I've simulated the construction of a 65K HTML report. In this case, there is a repeatable 6-7% performance drop (~685μs to 735μs on my last run). Upon asking on Slack, Valentin Churavy asked if I could try to find a minimal, repeatable test case.

The following test case that simulates a critical section, there is an increased runtime of about 3% (36.2μs to 34.9μs on my last run) from Julia 1.5.3 (788b2c7) to Julia 1.6.0-DEV.1722 (599d329).

using BenchmarkTools
BenchmarkTools.DEFAULT_PARAMETERS.seconds = 20

mutable struct SText{T}
    content::T
end

Base.print(io::IO, t::SText) = t.content(io)
    
stext(idx::Int) =
    return SText() do io::IO
             print(io, "<html><body><span>Womble #")
             print(io, idx)
             print(io, " reporting for duty!</span></body</html>")
          end

function srun()
    io = IOBuffer()
    for i in 1:100
       print(io, stext(i))
    end
end

println("stext ", @benchmark srun())

julia> versioninfo()
Julia Version 1.6.0-DEV.1722
Commit 599d3299c9 (2020-12-09 09:00 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: Intel(R) Celeron(R) CPU J3455 @ 1.50GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.0 (ORCJIT, goldmont)

julia> versioninfo()
Julia Version 1.5.3
Commit 788b2c77c1 (2020-11-09 13:37 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Celeron(R) CPU J3455 @ 1.50GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, goldmont)

If SText looks familiar -- it's because it is a replica of Docs.Text. I've got a secondary question... why is {T} needed here, if I remove it and use content::Function the performance hit is substantial. Moreover, when I delete only mutable in my broader test case... performance drops almost 2% (on same version of Julia). With this test case, swapping out {T} gives a good 5% or more drop in performance. It's hard to see the drop in performance when mutable is removed... but I'm pretty sure it's there, just tiny. I don't know what I'm actually doing here... am I making a brand new function for each request? Is that bad?

Anyway. I hope this is useful. Julia is awesome. The slow-down is also quite acceptable. It was just an outlier in otherwise amazing speedups and I thought 'yall might want to know about it.
@JeffBezanson
Copy link
Sponsor Member

I remove it and use content::Function the performance hit is substantial

That's because the t.content() call inside print becomes a dynamic dispatch; we no longer specialize on it.

This does make a new function object for each request, but that does not have much overhead, since they can all share the same code (i.e. there is only a single "type" of function, with multiple instances).

@clarkevans
Copy link
Member Author

clarkevans commented Dec 17, 2020 •
edited
Loading

@JeffBezanson Unfortunately, the behavior of this code snippet doesn't fully reflect the behavior of the broader application -- it just demonstrates a case where code is slower on Julia 1.6 development branch.

(removed stuff about mutable structs)

@clarkevans
Copy link
Member Author

clarkevans commented Dec 17, 2020 •
edited
Loading

@oscardssmith Observed on Slack that this particular example might be of interest to PR #37849

@JeffBezanson JeffBezanson added the performance Must go faster label Dec 17, 2020
@clarkevans
Copy link
Member Author

@JeffBezanson I posted a separate, perhaps simpler issue #38936 regarding mutable structs. I think it's intimately related to this one. So, it's OK if this one is closed.

@clarkevans clarkevans mentioned this issue Dec 18, 2020
Open
@clarkevans
Copy link
Member Author

I'm closing this ticket since I think #38936 is a more concrete example. Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
performance Must go faster
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@clarkevans @JeffBezanson