effects: add effects analysis for array construction #46015

aviatesk · 2022-07-13T09:16:32Z

This commit implements an infrastructure to analyze effects of
:foreigncall expression, and especially implements effects
analysis for array constructions.

Some improvements:

julia> @noinline construct_array(@nospecialize(T), args...) = Array{T}(undef, args...);

julia> function check_dims(T, dims)
           construct_array(T, dims...)
           return  nothing
       end;

julia> code_typed() do
           check_dims(Int, (1,2,3))
       end

diff --git a/_master b/_pr
index b0ed0eaac1..38e2d3553d 100644
--- a/_master
+++ b/_pr
@@ -1,6 +1,4 @@
 1-element Vector{Any}:
  CodeInfo(
-1 ─      invoke Main.construct_array(Int64::Any, -1::Int64, 2::Vararg{Int64}, 3)::Any
-│   %2 = Main.nothing::Nothing
-└──      return %2
+1 ─     return nothing
 ) => Nothing

julia> code_typed() do
          check_dims(Int, (-1,2,3))
      end

diff --git a/_master b/_pr
index b0ed0eaac1..e5e724db10 100644
--- a/_master
+++ b/_pr
@@ -1,6 +1,5 @@
 1-element Vector{Any}:
  CodeInfo(
-1 ─      invoke Main.construct_array(Int64::Any, -1::Int64, 2::Vararg{Int64}, 3)::Any
-│   %2 = Main.nothing::Nothing
-└──      return %2
-) => Nothing
+1 ─     invoke Main.check_dims(Main.Int::Type, (-1, 2, 3)::Tuple{Int64, Int64, Int64})::Union{}
+└──     unreachable
+) => Union{}

oscardssmith · 2022-07-13T15:51:36Z

Can you check whether this improves the time for allocating small arrays Iref
#45160)? On master, I'm seeing

julia> @btime Ref(Int);
  4.740 ns (1 allocation: 16 bytes)

julia> @btime Int[];
  21.435 ns (1 allocation: 64 bytes)

aviatesk · 2022-07-13T16:06:59Z

It shouldn't impact the time to allocate arrays. This is mostly for dead code elimination.

Keno · 2022-07-13T21:23:29Z

I would prefer if these used the existing capability to annotate effects on :foreigncalls rather than hacking this into abstract interpretation.

aviatesk · 2022-07-14T00:33:49Z

I would prefer if these used the existing capability to annotate effects on :foreigncalls rather than hacking this into abstract interpretation.

I found something like

Array{T,1}(::UndefInitializer, m::Int) where {T} = (@_total_but_inconsistent_meta;
    ccall(:jl_alloc_array_1d, Array{T,1}, (Any, Int), Array{T,1}, m))

is wrong as Array{Int,1}(undef, -1) for example throws, but I still wanted to make it :nothrow in order to make it is_removable_if_unused.
To Implement the analysis in abstract interpretation has another benefit that we can taint it as consistent = TRISTATE_UNKNOWN letting adjust_effects to refine the tainted :consistent-cy to ALWAYS_TRUE when the constructed array is never returned (#45701).

aviatesk · 2022-07-14T03:23:00Z

@nanosoldier runbenchmarks("inference", vs=":master")

nanosoldier · 2022-07-14T04:08:15Z

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here.

aviatesk · 2022-07-15T08:52:00Z

@nanosoldier runbenchmarks("inference", vs=":master")

nanosoldier · 2022-07-15T09:37:20Z

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here.

aviatesk · 2022-07-15T09:42:04Z

@nanosoldier runbenchmarks("inference", vs=":master")

nanosoldier · 2022-07-15T10:27:19Z

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here.

aviatesk · 2022-07-16T01:01:52Z

@nanosoldier runbenchmarks("inference", vs=":master")

nanosoldier · 2022-07-16T01:47:07Z

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here.

aviatesk · 2022-08-02T22:51:34Z

@nanosoldier runtests(["ASE", "CairoMakie", "CountdownNumbers", "DrelTools", "GraphNeuralNetworks", "IncrementalPruning", "KernelEstimator", "PoreMatMod", "ReinforcementLearningExperiments", "RetroCap", "StrBase", "SunAsAStar", "TopoPlots", "TuringGLM", "AbstractLogic", "FlameGraphs", "Folds", "GraphMLDatasets", "HYPRE", "HighDimPDE", "InformationGeometry", "Lux", "ProfileView", "StenoGraphs", "StressTest", "COPT", "Pidfile", "SBML"], vs = ":master")

nanosoldier · 2022-08-02T22:51:42Z

Something went wrong when running your job:

Base.IOError("unlink(\"/tmp/jl_c8vjoA\"): no such file or directory (ENOENT)", -2)

cc @maleadt

@noinline

This commit implements an infrastructure to analyze effects of `:foreigncall` expression, and especially implements effects analysis for array constructions. Some improvements: ```julia julia> @noinline construct_array(@nospecialize(T), args...) = Array{T}(undef, args...); julia> function check_dims(T, dims) construct_array(T, dims...) return nothing end; ``` ```julia julia> code_typed() do check_dims(Int, (1,2,3)) end ``` ```diff diff --git a/_master b/_pr index b0ed0eaac1..38e2d3553d 100644 --- a/_master +++ b/_pr @@ -1,6 +1,4 @@ 1-element Vector{Any}: CodeInfo( -1 ─ invoke Main.construct_array(Int64::Any, -1::Int64, 2::Vararg{Int64}, 3)::Any -│ %2 = Main.nothing::Nothing -└── return %2 +1 ─ return nothing ) => Nothing ``` ```julia julia> code_typed() do check_dims(Int, (-1,2,3)) end ``` ```diff diff --git a/_master b/_pr index b0ed0eaac1..e5e724db10 100644 --- a/_master +++ b/_pr @@ -1,6 +1,5 @@ 1-element Vector{Any}: CodeInfo( -1 ─ invoke Main.construct_array(Int64::Any, -1::Int64, 2::Vararg{Int64}, 3)::Any -│ %2 = Main.nothing::Nothing -└── return %2 -) => Nothing +1 ─ invoke Main.check_dims(Main.Int::Type, (-1, 2, 3)::Tuple{Int64, Int64, Int64})::Union{} +└── unreachable +) => Union{} ```

aviatesk · 2022-08-05T06:31:34Z

Test failures look unrelated.

DilumAluthge · 2022-08-05T08:42:00Z

@aviatesk It looks like this PR introduced some new compiler/effects test failures on i686-linux-gnu.

This reverts commit 01c0778.

@noinline

This commit implements an infrastructure to analyze effects of `:foreigncall` expression, and especially implements effects analysis for array constructions. Some improvements: ```julia julia> @noinline construct_array(@nospecialize(T), args...) = Array{T}(undef, args...); julia> function check_dims(T, dims) construct_array(T, dims...) return nothing end; ``` ```julia julia> code_typed() do check_dims(Int, (1,2,3)) end ``` ```diff diff --git a/_master b/_pr index b0ed0eaac1..38e2d3553d 100644 --- a/_master +++ b/_pr @@ -1,6 +1,4 @@ 1-element Vector{Any}: CodeInfo( -1 ─ invoke Main.construct_array(Int64::Any, -1::Int64, 2::Vararg{Int64}, 3)::Any -│ %2 = Main.nothing::Nothing -└── return %2 +1 ─ return nothing ) => Nothing ``` ```julia julia> code_typed() do check_dims(Int, (-1,2,3)) end ``` ```diff diff --git a/_master b/_pr index b0ed0eaac1..e5e724db10 100644 --- a/_master +++ b/_pr @@ -1,6 +1,5 @@ 1-element Vector{Any}: CodeInfo( -1 ─ invoke Main.construct_array(Int64::Any, -1::Int64, 2::Vararg{Int64}, 3)::Any -│ %2 = Main.nothing::Nothing -└── return %2 -) => Nothing +1 ─ invoke Main.check_dims(Main.Int::Type, (-1, 2, 3)::Tuple{Int64, Int64, Int64})::Union{} +└── unreachable +) => Union{} ```

…ng#46015)" This reverts commit 01c0778.

@noinline

This commit implements an infrastructure to analyze effects of `:foreigncall` expression, and especially implements effects analysis for array constructions. Some improvements: ```julia julia> @noinline construct_array(@nospecialize(T), args...) = Array{T}(undef, args...); julia> function check_dims(T, dims) construct_array(T, dims...) return nothing end; ``` ```julia julia> code_typed() do check_dims(Int, (1,2,3)) end ``` ```diff diff --git a/_master b/_pr index b0ed0eaac1..38e2d3553d 100644 --- a/_master +++ b/_pr @@ -1,6 +1,4 @@ 1-element Vector{Any}: CodeInfo( -1 ─ invoke Main.construct_array(Int64::Any, -1::Int64, 2::Vararg{Int64}, 3)::Any -│ %2 = Main.nothing::Nothing -└── return %2 +1 ─ return nothing ) => Nothing ``` ```julia julia> code_typed() do check_dims(Int, (-1,2,3)) end ``` ```diff diff --git a/_master b/_pr index b0ed0eaac1..e5e724db10 100644 --- a/_master +++ b/_pr @@ -1,6 +1,5 @@ 1-element Vector{Any}: CodeInfo( -1 ─ invoke Main.construct_array(Int64::Any, -1::Int64, 2::Vararg{Int64}, 3)::Any -│ %2 = Main.nothing::Nothing -└── return %2 -) => Nothing +1 ─ invoke Main.check_dims(Main.Int::Type, (-1, 2, 3)::Tuple{Int64, Int64, Int64})::Union{} +└── unreachable +) => Union{} ```

…ng#46015)" This reverts commit 01c0778.

aviatesk force-pushed the avi/array-effects branch from 8688442 to 9af7f1c Compare July 13, 2022 09:30

aviatesk requested review from Keno and vtjnash and removed request for Keno July 13, 2022 09:37

jakobnissen mentioned this pull request Jul 13, 2022

Allocating empty array is 4x slower than Java #45160

Closed

aviatesk force-pushed the avi/array-effects branch from 9af7f1c to ea377eb Compare July 13, 2022 17:11

aviatesk force-pushed the avi/array-effects branch from ea377eb to ff38b41 Compare July 14, 2022 03:22

aviatesk force-pushed the avi/array-effects branch from ff38b41 to f3a13af Compare July 15, 2022 08:45

aviatesk force-pushed the avi/array-effects branch from f3a13af to 416f0f2 Compare July 16, 2022 01:00

aviatesk force-pushed the avi/array-effects branch 2 times, most recently from 15ca11c to db391cc Compare August 1, 2022 02:58

aviatesk force-pushed the avi/array-effects branch 2 times, most recently from 516ee0a to 23a3935 Compare August 4, 2022 05:09

aviatesk force-pushed the avi/array-effects branch from 23a3935 to 6dc0ff5 Compare August 4, 2022 23:42

aviatesk merged commit 01c0778 into master Aug 5, 2022

aviatesk deleted the avi/array-effects branch August 5, 2022 06:31

DilumAluthge added a commit that referenced this pull request Aug 6, 2022

Revert "effects: add effects analysis for array construction (#46015)"

166231f

This reverts commit 01c0778.

DilumAluthge mentioned this pull request Aug 6, 2022

Revert "effects: add effects analysis for array construction" #46269

Merged

ffucci pushed a commit to ffucci/julia that referenced this pull request Aug 11, 2022

Revert "effects: add effects analysis for array construction (JuliaLa…

1c30c50

…ng#46015)" This reverts commit 01c0778.

pcjentsch pushed a commit to pcjentsch/julia that referenced this pull request Aug 18, 2022

Revert "effects: add effects analysis for array construction (JuliaLa…

0e8955d

…ng#46015)" This reverts commit 01c0778.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

effects: add effects analysis for array construction #46015

effects: add effects analysis for array construction #46015

aviatesk commented Jul 13, 2022

oscardssmith commented Jul 13, 2022 •

edited

Loading

aviatesk commented Jul 13, 2022

Keno commented Jul 13, 2022

aviatesk commented Jul 14, 2022

aviatesk commented Jul 14, 2022

nanosoldier commented Jul 14, 2022

aviatesk commented Jul 15, 2022

nanosoldier commented Jul 15, 2022

aviatesk commented Jul 15, 2022

nanosoldier commented Jul 15, 2022

aviatesk commented Jul 16, 2022

nanosoldier commented Jul 16, 2022

aviatesk commented Aug 2, 2022

nanosoldier commented Aug 2, 2022

aviatesk commented Aug 5, 2022

DilumAluthge commented Aug 5, 2022

effects: add effects analysis for array construction #46015

effects: add effects analysis for array construction #46015

Conversation

aviatesk commented Jul 13, 2022

oscardssmith commented Jul 13, 2022 • edited Loading

aviatesk commented Jul 13, 2022

Keno commented Jul 13, 2022

aviatesk commented Jul 14, 2022

aviatesk commented Jul 14, 2022

nanosoldier commented Jul 14, 2022

aviatesk commented Jul 15, 2022

nanosoldier commented Jul 15, 2022

aviatesk commented Jul 15, 2022

nanosoldier commented Jul 15, 2022

aviatesk commented Jul 16, 2022

nanosoldier commented Jul 16, 2022

aviatesk commented Aug 2, 2022

nanosoldier commented Aug 2, 2022

aviatesk commented Aug 5, 2022

DilumAluthge commented Aug 5, 2022

oscardssmith commented Jul 13, 2022 •

edited

Loading