-
-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
initial slowdowns in every fresh REPL session #51532
Comments
To save you the bisecting: #51399 The goal is to be able to upgrade standard libraries during a release. For that they can't be "baked" into the system image. That being said 2-4 should not have much more latency than before. If you want to investigate I recommend using Tracy to record a session. |
3 sounds like the tab completion stuff isn't getting precompiled properly. Can you share more information on your machine. versioninfo etc |
Ah, sorry, forgot to paste versioninfo above: julia> versioninfo()
Julia Version 1.11.0-DEV.585
Commit 64fc7db0556 (2023-10-01 11:54 UTC)
Build Info:
Official https://julialang.org/ release
Platform Info:
OS: Linux (x86_64-linux-gnu)
CPU: 8 × AMD Ryzen 3 5300U with Radeon Graphics
WORD_SIZE: 64
LLVM: libLLVM-15.0.7 (ORCJIT, znver2)
Threads: 1 on 8 virtual cores Not sure what other info is relevant. |
In case it happens to be useful, I ran Julia with precompile(Tuple{typeof(Base.Broadcast._getindex), Tuple{UInt8}, Int64})
precompile(Tuple{typeof(Base.promoteV), Type})
precompile(Tuple{Base.var"#1044#1046"{Bool, Symbol, Bool}, Module})
precompile(Tuple{Type{REPL.Terminals.TTYTerminal}, String, Base.TTY, Base.TTY, Base.TTY})
precompile(Tuple{typeof(Core.kwcall), NamedTuple{(:short,), Tuple{Bool}}, typeof(Base.banner), REPL.Terminals.TTYTerminal})
precompile(Tuple{typeof(Base.get), Base.TTY, Symbol, Bool})
precompile(Tuple{typeof(Base.getproperty), REPL.Terminals.TTYTerminal, Symbol})
precompile(Tuple{Type{REPL.LineEditREPL}, REPL.Terminals.TextTerminal, Bool, Bool})
precompile(Tuple{typeof(Base.setproperty!), REPL.Options, Symbol, Bool})
precompile(Tuple{typeof(Base.setproperty!), REPL.LineEditREPL, Symbol, Bool})
precompile(Tuple{Type{REPL.REPLDisplay{Repl} where Repl<:REPL.AbstractREPL}, REPL.LineEditREPL})
precompile(Tuple{typeof(Base.__atreplinit), REPL.LineEditREPL})
precompile(Tuple{typeof(REPL.run_repl), REPL.AbstractREPL, Any})
precompile(Tuple{typeof(Core.kwcall), NamedTuple{names, T} where T<:Tuple where names, typeof(REPL.start_repl_backend), REPL.REPLBackend, Any})
precompile(Tuple{typeof(Base.structdiff), NamedTuple{(:get_module,), Tuple{REPL.var"#63#69"{REPL.LineEditREPL}}}, Type{NamedTuple{(:get_module,), T} where T<:Tuple}})
precompile(Tuple{Base.var"#1045#1047", REPL.REPLBackend})
precompile(Tuple{Type{REPL.LineEdit.Modifiers}, Bool})
precompile(Tuple{Type{NamedTuple{(:pass_empty, :suppress_on_semicolon), T} where T<:Tuple}, Tuple{Bool, Bool}})
precompile(Tuple{Type{NamedTuple{(:read, :write, :create), T} where T<:Tuple}, Tuple{Bool, Bool, Bool}})
precompile(Tuple{Type{Pair{A, B} where B where A}, String, Function})
precompile(Tuple{Type{Pair{A, B} where B where A}, Char, Function})
precompile(Tuple{Type{Pair{A, B} where B where A}, Int64, REPL.LineEdit.KeyAlias})
precompile(Tuple{typeof(Base.getproperty), Pair{String, REPL.LineEdit.var"#50#81"}, Symbol})
precompile(Tuple{Type{REPL.LineEdit.InputAreaState}, Int64, Int64})
precompile(Tuple{REPL.var"#64#70"{REPL.LineEditREPL, REPL.REPLBackendRef}})
precompile(Tuple{typeof(Base.structdiff), NamedTuple{(:prompt_prefix, :prompt_suffix, :repl, :complete, :on_enter), Tuple{String, typeof(Base.input_color), REPL.LineEditREPL, REPL.REPLCompletionProvider, typeof(REPL.return_callback)}}, Type{NamedTuple{(:prompt_prefix, :prompt_suffix, :output_prefix, :output_prefix_prefix, :output_prefix_suffix, :keymap_dict, :repl, :complete, :on_enter, :on_done, :hist, :sticky), T} where T<:Tuple}})
precompile(Tuple{Type{REPL.LineEdit.Prompt}, Union{Function, String}, Union{Function, String}, Union{Function, String}, Union{Function, String}, Union{Function, String}, Union{Function, String}, Base.Dict{Char, Any}, Union{Nothing, REPL.AbstractREPL}, REPL.LineEdit.CompletionProvider, Function, Function, REPL.LineEdit.HistoryProvider, Bool})
precompile(Tuple{typeof(Base.structdiff), NamedTuple{(:prompt_prefix, :prompt_suffix, :repl, :complete, :on_done), Tuple{String, typeof(Base.input_color), REPL.LineEditREPL, REPL.REPLCompletionProvider, REPL.var"#do_respond#82"{Bool, Bool, REPL.var"#92#104"{REPL.LineEditREPL}, REPL.LineEditREPL, REPL.LineEdit.Prompt}}}, Type{NamedTuple{(:prompt_prefix, :prompt_suffix, :output_prefix, :output_prefix_prefix, :output_prefix_suffix, :keymap_dict, :repl, :complete, :on_enter, :on_done, :hist, :sticky), T} where T<:Tuple}})
precompile(Tuple{typeof(Base.structdiff), NamedTuple{(:prompt_prefix, :prompt_suffix, :repl, :complete, :on_done, :sticky), Tuple{String, typeof(Base.input_color), REPL.LineEditREPL, REPL.ShellCompletionProvider, REPL.var"#do_respond#82"{Bool, Bool, REPL.var"#93#105"{REPL.LineEditREPL}, REPL.LineEditREPL, REPL.LineEdit.Prompt}, Bool}}, Type{NamedTuple{(:prompt_prefix, :prompt_suffix, :output_prefix, :output_prefix_prefix, :output_prefix_suffix, :keymap_dict, :repl, :complete, :on_enter, :on_done, :hist, :sticky), T} where T<:Tuple}})
precompile(Tuple{typeof(Base.convert), Type{Array{Symbol, 1}}, Array{UInt8, 1}})
precompile(Tuple{typeof(Base.setindex!), Base.Dict{Any, Any}, Any, Int64})
precompile(Tuple{typeof(Base.hashindex), String, Int64})
precompile(Tuple{typeof(Base.delete!), Base.Set{Any}, String})
precompile(Tuple{typeof(Base.:(==)), Char, String})
precompile(Tuple{typeof(Base.prepend!), Array{Base.Dict{Any, Any}, 1}, Array{Base.Dict{Any, Any}, 1}})
precompile(Tuple{typeof(REPL.LineEdit.init_state), Any, REPL.LineEdit.Prompt})
precompile(Tuple{Type{REPL.LineEdit.PromptState}, REPL.Terminals.AbstractTerminal, REPL.LineEdit.Prompt, Base.GenericIOBuffer{Array{UInt8, 1}}, Symbol, Union{Nothing, String}, Array{Base.GenericIOBuffer{Array{UInt8, 1}}, 1}, Int64, REPL.LineEdit.InputAreaState, Int64, Base.Threads.SpinLock, Float64, Float64, Union{Nothing, Base.Timer}})
precompile(Tuple{typeof(REPL.LineEdit.init_state), Any, REPL.LineEdit.HistoryPrompt})
precompile(Tuple{typeof(REPL.LineEdit.init_state), Any, REPL.LineEdit.PrefixHistoryPrompt})
precompile(Tuple{typeof(REPL.LineEdit.run_interface), REPL.Terminals.TextTerminal, REPL.LineEdit.ModalInterface, REPL.LineEdit.MIState})
precompile(Tuple{typeof(Base.reseteof), REPL.Terminals.TTYTerminal})
precompile(Tuple{typeof(REPL.Terminals.raw!), REPL.Terminals.TTYTerminal, Bool})
precompile(Tuple{typeof(REPL.Terminals.enable_bracketed_paste), REPL.Terminals.TTYTerminal})
precompile(Tuple{typeof(REPL.LineEdit.activate), REPL.LineEdit.ModalInterface, REPL.LineEdit.MIState, REPL.Terminals.AbstractTerminal, REPL.Terminals.TextTerminal})
precompile(Tuple{typeof(REPL.LineEdit.activate), REPL.LineEdit.TextInterface, REPL.LineEdit.MIState, REPL.Terminals.AbstractTerminal, REPL.Terminals.TextTerminal})
precompile(Tuple{typeof(Base.setproperty!), REPL.LineEdit.PromptState, Symbol, REPL.LineEdit.InputAreaState})
precompile(Tuple{typeof(REPL.LineEdit.refresh_line), Union{Base.GenericIOBuffer{Array{UInt8, 1}}, REPL.LineEdit.MIState, REPL.LineEdit.ModeState}, REPL.Terminals.AbstractTerminal})
precompile(Tuple{typeof(REPL.LineEdit.refresh_multi_line), REPL.Terminals.UnixTerminal, Any})
precompile(Tuple{REPL.LineEdit.var"##refresh_multi_line#17", Base.Pairs{Symbol, V, Tuple{Vararg{Symbol, N}}, NamedTuple{names, T}} where T<:Tuple{Vararg{Any, N}} where names where N where V, typeof(REPL.LineEdit.refresh_multi_line), REPL.Terminals.UnixTerminal, Any})
precompile(Tuple{typeof(Base.getproperty), Base.MappingRF{Base.ExtremaMap{typeof(Base.identity)}, Base.BottomRF{typeof(Base._extrema_rf)}}, Symbol})
precompile(Tuple{typeof(Base.getproperty), Base.BottomRF{typeof(Base._extrema_rf)}, Symbol})
precompile(Tuple{typeof(Base.iszero), Float64})
precompile(Tuple{typeof(REPL.LineEdit.refresh_multi_line), REPL.Terminals.TerminalBuffer, REPL.Terminals.UnixTerminal, Union{REPL.LineEdit.PrefixSearchState, REPL.LineEdit.PromptState}})
precompile(Tuple{typeof(REPL.Terminals.clear_line), REPL.Terminals.TerminalBuffer})
precompile(Tuple{typeof(Base.write), Base.GenericIOBuffer{Array{UInt8, 1}}, String})
precompile(Tuple{typeof(REPL.Terminals.width), REPL.Terminals.TTYTerminal})
precompile(Tuple{typeof(REPL.Terminals.height), REPL.Terminals.TTYTerminal})
precompile(Tuple{typeof(REPL.Terminals.hascolor), REPL.Terminals.TTYTerminal})
precompile(Tuple{typeof(REPL.LineEdit.write_prompt), REPL.Terminals.AbstractTerminal, REPL.LineEdit.PromptState, Bool})
precompile(Tuple{typeof(Base.write), REPL.Terminals.TerminalBuffer, String})
precompile(Tuple{typeof(REPL.LineEdit.write_prompt), IO, Union{AbstractString, Function}, Bool})
precompile(Tuple{REPL.var"#89#90"{REPL.LineEditREPL, String}})
precompile(Tuple{typeof(Base.write), REPL.Terminals.TTYTerminal, Array{UInt8, 1}})
precompile(Tuple{typeof(REPL.LineEdit.keymap), REPL.LineEdit.PromptState, REPL.LineEdit.Prompt})
precompile(Tuple{typeof(REPL.LineEdit.match_input), Base.Dict{Char, V} where V, Union{Nothing, REPL.LineEdit.MIState}})
precompile(Tuple{typeof(REPL.LineEdit.terminal), REPL.LineEdit.PromptState})
precompile(Tuple{typeof(REPL.LineEdit.match_input), Base.Dict{Char, V} where V, Union{Nothing, REPL.LineEdit.MIState}, Union{Base.GenericIOBuffer{Array{UInt8, 1}}, REPL.Terminals.AbstractTerminal}, Array{Char, 1}})
precompile(Tuple{typeof(REPL.LineEdit.match_input), Base.Dict{Char, V} where V, Union{Nothing, REPL.LineEdit.MIState}, Union{Base.GenericIOBuffer{Array{UInt8, 1}}, REPL.Terminals.AbstractTerminal}, Array{Char, 1}, Base.Dict{Char, V} where V})
precompile(Tuple{typeof(Base.eof), REPL.Terminals.TTYTerminal})
precompile(Tuple{Base.var"#readcb_specialized#759", Base.TTY, Int64, UInt64})
precompile(Tuple{typeof(Base.read), REPL.Terminals.TTYTerminal, Type{Char}})
precompile(Tuple{typeof(Base.read), Base.TTY, Type{UInt8}})
precompile(Tuple{typeof(Base.get), Base.Dict{Char, Any}, Char, Nothing})
precompile(Tuple{typeof(REPL.LineEdit.match_input), Function, Union{Nothing, REPL.LineEdit.MIState}, Any, Array{Char, 1}, Any})
precompile(Tuple{typeof(REPL.LineEdit.keymap_data), REPL.LineEdit.PromptState, REPL.LineEdit.Prompt})
precompile(Tuple{REPL.LineEdit.var"#27#28"{REPL.LineEdit.var"#139#195", String}, Any, Any})
precompile(Tuple{typeof(Base.isempty), Nothing})
precompile(Tuple{typeof(Base.isequal), UInt8})
precompile(Tuple{REPL.LineEdit.var"#139#195", REPL.LineEdit.MIState, Any, Union{Char, Base.SubString{String}, String}})
precompile(Tuple{typeof(REPL.LineEdit.edit_insert), REPL.LineEdit.MIState, Any})
precompile(Tuple{typeof(REPL.LineEdit.region_active), REPL.LineEdit.PromptState})
precompile(Tuple{typeof(REPL.LineEdit.activate_region), REPL.LineEdit.PromptState, Symbol})
precompile(Tuple{typeof(Base.write), REPL.Terminals.TTYTerminal, String})
precompile(Tuple{typeof(REPL.LineEdit._buffer), REPL.LineEdit.PromptState})
precompile(Tuple{typeof(Base.getproperty), REPL.LineEdit.TextInterface, Symbol})
precompile(Tuple{Type{NamedTuple{(:raise, :depwarn), T} where T<:Tuple}, Tuple{Bool, Bool}})
precompile(Tuple{Type{NamedTuple{(:aggressive_constant_propagation, :unoptimize_throw_blocks), T} where T<:Tuple}, Tuple{Bool, Bool}})
precompile(Tuple{typeof(Core.kwcall), NamedTuple{(:aggressive_constant_propagation, :unoptimize_throw_blocks), Tuple{Bool, Bool}}, Type{Core.Compiler.InferenceParams}})
precompile(Tuple{Base.var"##s129#248", Vararg{Any, 5}})
precompile(Tuple{typeof(Base._nt_names), Type{NamedTuple{(:scratch,), Tuple{Array{Symbol, 1}}}}})
precompile(Tuple{Type{NamedTuple{(:all, :imported), T} where T<:Tuple}, Tuple{Bool, Bool}})
precompile(Tuple{Type{NamedTuple{(:c_start, :c_end), T} where T<:Tuple}, Tuple{Char, Char}})
precompile(Tuple{typeof(Base.in), Array{Char, 1}})
precompile(Tuple{typeof(Base.indexed_iterate), Tuple{Nothing, Nothing, Nothing}, Int64})
precompile(Tuple{typeof(Base.keys), Base.Dict{String, String}})
precompile(Tuple{Type{NamedTuple{(:raise,), T} where T<:Tuple}, Tuple{Bool}})
precompile(Tuple{typeof(Base._nt_names), Type{NamedTuple{(:scratch,), Tuple{Array{REPL.REPLCompletions.Completion, 1}}}}})
precompile(Tuple{Type{Pair{A, B} where B where A}, Base.Regex, String})
precompile(Tuple{Type{NamedTuple{(:shell_escape,), T} where T<:Tuple}, Tuple{Bool}})
precompile(Tuple{Type{NamedTuple{(:join, :sort), T} where T<:Tuple}, Tuple{Bool, Bool}})
precompile(Tuple{typeof(Base.getproperty), Base.Order.By{REPL.REPLCompletions.var"#46#49", Base.Order.ForwardOrdering}, Symbol})
precompile(Tuple{typeof(Base.getproperty), Base.Order.Lt{Base.Sort.var"#26#27"{Base.Order.By{REPL.REPLCompletions.var"#46#49", Base.Order.ForwardOrdering}}}, Symbol})
precompile(Tuple{Type{NamedTuple{(:string_escape,), T} where T<:Tuple}, Tuple{Bool}})
precompile(Tuple{typeof(Base.getproperty), Base.Order.By{REPL.REPLCompletions.var"#47#50", Base.Order.ForwardOrdering}, Symbol})
precompile(Tuple{typeof(Base.getproperty), Base.Order.Lt{Base.Sort.var"#26#27"{Base.Order.By{REPL.REPLCompletions.var"#47#50", Base.Order.ForwardOrdering}}}, Symbol})
precompile(Tuple{typeof(REPL.LineEdit.complete_line), REPL.REPLCompletionProvider, REPL.LineEdit.PromptState, Module})
precompile(Tuple{typeof(Base.incomplete_tag), Symbol})
precompile(Tuple{typeof(Base.filter!), REPL.REPLCompletions.var"#11#12"{Base.Returns{Bool}, Module, Symbol}, Array{Symbol, 1}})
precompile(Tuple{REPL.REPLCompletions.var"#3#8", Base.GenericIOBuffer{Array{UInt8, 1}}, Symbol})
precompile(Tuple{typeof(Base.in!), REPL.REPLCompletions.ModuleCompletion, Base.Set{REPL.REPLCompletions.Completion}})
precompile(Tuple{typeof(Base.Order.lt), Base.Order.By{typeof(REPL.REPLCompletions.completion_text), Base.Order.ForwardOrdering}, REPL.REPLCompletions.ModuleCompletion, REPL.REPLCompletions.ModuleCompletion})
precompile(Tuple{typeof(REPL.REPLCompletions.completion_text), REPL.REPLCompletions.ModuleCompletion})
precompile(Tuple{REPL.LineEdit.var"#27#28"{REPL.LineEdit.var"#117#173", String}, Any, Any})
precompile(Tuple{REPL.LineEdit.var"#117#173", REPL.LineEdit.MIState, Any, Vararg{Any}})
precompile(Tuple{typeof(REPL.LineEdit.is_region_active), REPL.LineEdit.PromptState})
precompile(Tuple{REPL.LineEdit.var"##refresh_multi_line#12", Base.Pairs{Symbol, V, Tuple{Vararg{Symbol, N}}, NamedTuple{names, T}} where T<:Tuple{Vararg{Any, N}} where names where N where V, typeof(REPL.LineEdit.refresh_multi_line), REPL.LineEdit.PromptState})
precompile(Tuple{typeof(Base.min), Float64, Float64})
precompile(Tuple{typeof(Base.setproperty!), REPL.LineEdit.PromptState, Symbol, Float64})
precompile(Tuple{typeof(Base.copymutable), Array{String, 1}})
precompile(Tuple{Type{NamedTuple{(:beeping,), T} where T<:Tuple}, Tuple{Bool}})
precompile(Tuple{typeof(Base.pairs), NamedTuple{(:beeping,), Tuple{Bool}}})
precompile(Tuple{REPL.LineEdit.var"#6#7"{Bool, REPL.LineEdit.PromptState, Float64, Array{String, 1}}})
precompile(Tuple{typeof(Core.kwcall), NamedTuple{names, T} where T<:Tuple where names, typeof(REPL.LineEdit.refresh_multi_line), REPL.Terminals.UnixTerminal, Any})
precompile(Tuple{typeof(Core.kwcall), NamedTuple{names, T} where T<:Tuple where names, typeof(REPL.LineEdit.refresh_multi_line), REPL.Terminals.TerminalBuffer, REPL.Terminals.UnixTerminal, Union{REPL.LineEdit.PrefixSearchState, REPL.LineEdit.PromptState}})
precompile(Tuple{typeof(Base.structdiff), NamedTuple{(:beeping,), Tuple{Bool}}, Type{NamedTuple{(:beeping,), T} where T<:Tuple}})
precompile(Tuple{REPL.LineEdit.var"#27#28"{REPL.LineEdit.var"#119#175", String}, Any, Any})
precompile(Tuple{REPL.LineEdit.var"#119#175", REPL.LineEdit.MIState, Any, Vararg{Any}})
precompile(Tuple{REPL.LineEdit.var"##edit_abort#113", Any, typeof(REPL.LineEdit.edit_abort), REPL.LineEdit.MIState, Bool})
precompile(Tuple{typeof(Base.println), REPL.Terminals.TTYTerminal})
precompile(Tuple{typeof(Base.write), Base.TTY, UInt8})
precompile(Tuple{typeof(REPL.Terminals.disable_bracketed_paste), REPL.Terminals.TTYTerminal})
precompile(Tuple{typeof(REPL.LineEdit.mode), REPL.LineEdit.PromptState})
precompile(Tuple{REPL.var"#do_respond#82"{Bool, Bool, REPL.var"#95#107"{REPL.LineEditREPL, REPL.REPLHistoryProvider}, REPL.LineEditREPL, REPL.LineEdit.Prompt}, REPL.LineEdit.MIState, Any, Bool})
precompile(Tuple{typeof(Base.indexed_iterate), Tuple{Nothing, Int64}, Int64})
precompile(Tuple{typeof(Base.indexed_iterate), Tuple{Nothing, Int64}, Int64, Int64})
precompile(Tuple{typeof(Base.print), Base.TTY, String})
precompile(Tuple{REPL.var"#62#68"{REPL.REPLBackendRef}})
precompile(Tuple{typeof(Base._atexit), Int32})
precompile(Tuple{REPL.var"#94#106"{REPL.REPLHistoryProvider}, Any}) |
Interesting doing the same locally for me is:
Which still is some, but not nearly as many.
Did you download the nightly or build locally? |
Do you have a |
Downloaded the nightly. I think the
entry.
I don't even have
Doesn't seem to change anything. |
I'm experiencing the same problems as @nsajko. The output of
|
The plot thickens... When I download the nightly from Julia I also just get the results from #51532 (comment) What is Could you also post |
|
On MacOS Silicon I see virtually the same with a local build and the downloaded nightly and both are immediate.
The only difference is
|
I just tired the latest 1.11 (590) and 1.10-beta2 and startups seem to be equally fast. This is both with and without startup.jl. I am really happy with how fast 1.10 comes up. I'd hate to lose that with 1.11. M2 Mac running Sonoma |
It seems I'm now able to reliably reproduce this from scratch (run Julia with
So it seems that a single |
Nice reduction! Can you try that again with steps 5 & 6 with |
So I can explain We allow Julia to load cache files generated at higher optimization levels. So once you "polluted" your cache with Now the question is: Why is precompilation at |
Even better when I invalidate The REPL precompilation process is quite special. In particular we spawn another process that we issue commands too: julia/stdlib/REPL/src/precompile.jl Lines 81 to 83 in 6a1af76
Looking at this with Tracy I saw that I strongly dislike this whole song and dance and wish that there was to do a way to do it reliably in process. |
IIRC, the reason it is done like this is mostly to avoid any side effects of the workloads from getting serialized into the sysimage. Maybe it can be changed though. |
5:
6:
|
I guess we should set up the |
I was thinking about that, but the challenge is that we want to allow some cachefiles. I.e. all of the repl dependencies... So we can't set it to a empty depot |
Set |
On nightly, but not on 1.10.0-beta2, I get slowdowns in each fresh REPL session.
Examples:
There's a noticeable delay from starting
julia
to being greeted with the standard messageThere's a noticeable delay from being greeted to getting the first
julia>
promptSuppose I mash some keys on the keyboard, e.g.,
qwert
: there's first a delay until the initialq
is displayed, then another delay until the rest,wert
, is displayed. After those two delays, I can type other letters with no perceptible delay.There's a noticeable delay from the first time I press ENTER to getting the next
julia>
prompt. After that things seem to go smoothly.Maybe I try bisecting this later.
The text was updated successfully, but these errors were encountered: