Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

release-1.7: Backports for 1.7.0/1.7.0-rc3 #42765

Merged
merged 72 commits into from
Nov 13, 2021
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
72 commits
Select commit Hold shift + click to select a range
0ac9c68
change locks to avoid capturing pthread_self value
vtjnash Oct 7, 2021
976e63c
release-1.7: set VERSION to 1.7-rc2 (#42716)
KristofferC Oct 20, 2021
982d585
CI (Buildkite): fix the timeout for the `tester_linux` job that runs …
DilumAluthge Oct 21, 2021
e86ee1f
fix unique over a dimension to use `isequal` over `==` (#42737)
KristofferC Oct 21, 2021
4cbdf42
task: schedule sticky tasks correctly with _wait2 (#42750)
vtjnash Oct 22, 2021
f4df914
Distributed test suite: increase the timeout in the `poll_while` func…
DilumAluthge Oct 22, 2021
448072f
fix overflow and undeflow for @fastmath exp (#42747)
oscardssmith Oct 22, 2021
069861a
Disable `DL_LOAD_PATH` prepending for `@`-paths on Darwin (#42721)
staticfloat Oct 21, 2021
e0f980e
manually-cherry-picked: optimizer: eliminate excessive specialization…
aviatesk Oct 24, 2021
da71d29
manually-cherry-picked: optimizer: fix #42754, inline union-split con…
aviatesk Oct 24, 2021
c22889e
manually-cherry-picked: inference: `prioritize force_constant_prop` o…
aviatesk Aug 24, 2021
f662b0a
Merge pull request #42805 from JuliaLang/avi/backport-42766
aviatesk Oct 27, 2021
4bf1a0c
[release-1.7] Bump the Downloads stdlib from 5d00bdd (v1.5.1) to 26d7…
DilumAluthge Oct 28, 2021
532b3f8
Distributed test suite: if `Threads.nthreads() > 1`, skip certain tes…
DilumAluthge Oct 25, 2021
e3255ef
fix #41546, make `using` thread-safe (#41602)
JeffBezanson Oct 25, 2021
8869704
CI (Buildkite): make sure to hit ignore any unencrypted repo keys, re…
DilumAluthge Oct 26, 2021
84b7ded
reset `RandomDevice` file from `__init__` (#42537)
JeffBezanson Oct 27, 2021
21a2784
🤖 Bump the Statistics stdlib from 74897fe to 5256d57 (#42826)
DilumAluthgeBot Oct 28, 2021
901a3a5
Revert "fix #41546, make `using` thread-safe (#41602)"
KristofferC Oct 29, 2021
382129f
optimizer: fix #42840, the performance regression introduced by #4276…
aviatesk Oct 29, 2021
d7ccfd2
Backport PR #42152 to 1.7 (#42828)
barche Oct 31, 2021
17e60a2
CI: add TSAN to the sanitizers pipelines (#42444)
tkf Oct 4, 2021
3404ae5
CI (Buildkite): Update all rootfs images to the latest versions (#42802)
DilumAluthge Oct 28, 2021
c76de1e
Update Patchelf to 0.13
vchuravy Oct 25, 2021
3f030a7
CI (Buildkite): add a separate Buildkite job that runs the tests of t…
DilumAluthge Oct 31, 2021
1ef8f95
Optimize show(io::IO, m::Module) implementation. (#42773)
Sacha0 Oct 31, 2021
13310eb
no longer explicitly print some things in white (#42864)
KristofferC Oct 31, 2021
9aa1dc3
Fix RPATH of libLLVM.so on FreeBSD and Linux
vchuravy Oct 25, 2021
57f6f27
Merge pull request #42788 from JuliaLang/vc/freebsd_fix
vchuravy Oct 31, 2021
0c92bf2
Test suite: improve the warning that we print when we skip the `Profi…
DilumAluthge Oct 31, 2021
b6f1cbc
`choosetests`: preparations for setting up a separate Buildkite job t…
DilumAluthge Nov 1, 2021
b825b2d
use the httable key instead of the binding name in jl_module_names (#…
Pangoraw Nov 1, 2021
75961b2
handle some cases of null Task in segv handling (#42836)
vtjnash Nov 1, 2021
4b8fba2
MINSIGSTKSZ is no longer constant in glibc (#41860)
t-bltg Aug 12, 2021
5830b81
Update BinaryBuilder libuv to 2.0.1+4 (#41900)
ararslan Aug 18, 2021
bdcc62b
Update patchelf.mk to always use bzp2 (#42881)
vchuravy Nov 1, 2021
bc1e4ea
`choosetests`: `Pkg/pkg` is no longer relevant (#42890)
DilumAluthge Nov 2, 2021
4871167
CI (Buildkite): add Pkg, Artifacts, and LazyArtifacts to the list of …
DilumAluthge Nov 4, 2021
eb951d3
Eliminate `@loader_path/julia` element of `DL_LOAD_PATH` (#42930)
staticfloat Nov 4, 2021
6cc14ad
Distributed test suite: mark another test as thread-unsafe (#42941)
DilumAluthge Nov 5, 2021
6b08828
CI (GHA, Buildbot): temporarily, don't create the `linuxaarch64` Buil…
DilumAluthge Nov 5, 2021
1fee4a5
CI (Buildkite): increase the default timeouts (#42953)
DilumAluthge Nov 5, 2021
b7f4af8
CI (Buildkite): Fix the Buildkite once-daily scheduled job (#42976)
DilumAluthge Nov 7, 2021
6da57c2
CI (Buildkite): increase the timeouts for the Buildkite jobs that run…
DilumAluthge Nov 7, 2021
2716958
CI (GHA, Buildbot): Update the list of commit statuses created by the…
DilumAluthge Nov 7, 2021
242d851
CI (Buildkite): fix another bug in the once-daily scheduled Buildkite…
DilumAluthge Nov 7, 2021
f8ae06f
Cleanup `RPATH` settings (#42919)
staticfloat Nov 3, 2021
bf1f998
Partially revert #42919 (#42969)
ararslan Nov 6, 2021
f5954c1
External stdlibs: Move `*_URL` stdlib definitions into `.version` files
DilumAluthge Oct 2, 2021
d74571f
Remove some other uses of the unauthenticated `git:https://` protocol in Gi…
DilumAluthge Nov 2, 2021
ee25032
Fix libunwind segfaults from JIT frames on FreeBSD (#42970)
ararslan Nov 7, 2021
9723f82
Keep expected crash in misc test from producing a core dump (#42990)
ararslan Nov 8, 2021
bbe91db
inference: improve `TypeVar`/`Vararg` handling
aviatesk Oct 11, 2021
2a4d14f
eliminate unbound `TypeVar`s on `argtypes` construction
aviatesk Oct 15, 2021
643fef7
inference: eliminate more potential `widenconst(::TypeofVararg)` case…
aviatesk Nov 7, 2021
e8caa27
inference: relax type_more_complex for Type
vtjnash Nov 1, 2021
87000f3
tests: add help and reflection abilities to choosetests option parser…
vtjnash Oct 4, 2021
1679ea1
`choosetests`: add the `--force-net` option, which will throw an erro…
DilumAluthge Nov 1, 2021
3479d89
Running `make testall` should run all of the tests (#42904)
DilumAluthge Nov 7, 2021
41f4756
choosetests: return a namedtuple (instead of a tuple) (#42723)
DilumAluthge Oct 21, 2021
1dca956
choosetests: move testnames array at top-level (#40182)
rfourquet Jun 15, 2021
15e822c
bump to latest Pkg release-1.7
KristofferC Nov 10, 2021
52f36b8
CI (Buildkite): increase the timeout of the code coverage job (#42994)
DilumAluthge Nov 8, 2021
118344d
Fix segfault while profiling task switching (#42973)
tkf Nov 8, 2021
4c2229a
Timer: handle timeout correctly (#42854)
vtjnash Nov 8, 2021
82af540
Add missing USE_SYSTEM_LIBWHICH initialization (#43014)
haampie Nov 9, 2021
7617229
Add missing type parameter to TakeWhile (#42958)
pepijndevos Nov 9, 2021
40dca0f
`file` tests: in the `"tempname with parent"` test, temporarily unset…
DilumAluthge Nov 9, 2021
9e11179
CI (Buildkite): use a `TMPDIR` that is backed by real storage (#42995)
DilumAluthge Nov 10, 2021
8597a4b
Allow system versions of libwhich and libblastrampoline (#43000)
haampie Nov 8, 2021
9132a40
disallow unbalanced bidirectional formatting in strings and comments …
JeffBezanson Nov 11, 2021
5da2e44
Print symbols in `Tuple{:sym}` type with colons (#42999)
inkydragon Nov 10, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
manually-cherry-picked: inference: prioritize force_constant_prop o…
…ver `const_prop_entry_heuristic` (#41882)

Currently our constant-prop' heuristics work in the following way:
1. `const_prop_entry_heuristic`
2. `const_prop_argument_heuristic` & `const_prop_rettype_heuristic`
3. `force_const_prop` custom heuristic & `!const_prop_function_heuristic`
4. `MethodInstance` specialization and `const_prop_methodinstance_heuristic`

This PR changes it so that the step 1. now works like:

1. `force_const_prop` custom heuristic & `const_prop_entry_heuristic`

and the steps 2., 3. and 4. don't change

This change particularly allows us to more forcibly constant-propagate
for `getproperty` and `setproperty!`, and inline them more, e.g.:
```julia
mutable struct Foo
    val
    _::Int
end

function setter(xs)
    for x in xs
        x.val = nothing # `setproperty!` can be inlined with this PR
    end
end
```

It might be useful because now we can intervene into the constant-prop'
heuristic in a more reliable way with the `aggressive_constprop` interface.

I did the simple benchmark below, and it looks like this change doesn't
cause the latency problem for this particular example:
```zsh
~/julia master aviatesk@amdci2 6s
❯ ./usr/bin/julia -e '@time using Plots; @time plot(rand(10,3))'
  3.708500 seconds (7.28 M allocations: 506.128 MiB, 3.45% gc time, 1.13% compilation time)
  2.817794 seconds (3.45 M allocations: 195.127 MiB, 7.84% gc time, 53.76% compilation time)

~/julia avi/forceconstantprop aviatesk@amdci2 6s
❯ ./usr/bin/julia -e '@time using Plots; @time plot(rand(10,3))'
  3.622109 seconds (7.02 M allocations: 481.710 MiB, 4.19% gc time, 1.17% compilation time)
  2.863419 seconds (3.44 M allocations: 194.210 MiB, 8.02% gc time, 53.53% compilation time)
```
  • Loading branch information
aviatesk committed Oct 26, 2021
commit c22889e76cb9b7fd8a4710d9bf53e827aaa907e4
29 changes: 22 additions & 7 deletions base/compiler/abstractinterpretation.jl
Original file line number Diff line number Diff line change
Expand Up @@ -568,17 +568,27 @@ end
function maybe_get_const_prop_profitable(interp::AbstractInterpreter, result::MethodCallResult,
@nospecialize(f), argtypes::Vector{Any}, match::MethodMatch,
sv::InferenceState)
const_prop_entry_heuristic(interp, result, sv) || return nothing
if !InferenceParams(interp).ipo_constant_propagation
add_remark!(interp, sv, "[constprop] Disabled by parameter")
return nothing
end
method = match.method
force = force_const_prop(interp, f, method)
force || const_prop_entry_heuristic(interp, result, sv) || return nothing
nargs::Int = method.nargs
method.isva && (nargs -= 1)
if length(argtypes) < nargs
length(argtypes) < nargs && return nothing
if !(const_prop_argument_heuristic(interp, argtypes) || const_prop_rettype_heuristic(interp, result.rt))
add_remark!(interp, sv, "[constprop] Disabled by argument and rettype heuristics")
return nothing
end
const_prop_argument_heuristic(interp, argtypes) || const_prop_rettype_heuristic(interp, result.rt) || return nothing
allconst = is_allconst(argtypes)
force = force_const_prop(interp, f, method)
force || const_prop_function_heuristic(interp, f, argtypes, nargs, allconst) || return nothing
if !force
if !const_prop_function_heuristic(interp, f, argtypes, nargs, allconst)
add_remark!(interp, sv, "[constprop] Disabled by function heuristic")
return nothing
end
end
force |= allconst
mi = specialize_method(match, !force)
if mi === nothing
Expand All @@ -594,8 +604,13 @@ function maybe_get_const_prop_profitable(interp::AbstractInterpreter, result::Me
end

function const_prop_entry_heuristic(interp::AbstractInterpreter, result::MethodCallResult, sv::InferenceState)
call_result_unused(sv) && result.edgecycle && return false
return is_improvable(result.rt) && InferenceParams(interp).ipo_constant_propagation
if call_result_unused(sv) && result.edgecycle
add_remark!(interp, sv, "[constprop] Disabled by entry heuristic (edgecycle with unused result)")
return false
end
is_improvable(result.rt) && return true
add_remark!(interp, sv, "[constprop] Disabled by entry heuristic (unimprovable return type)")
return false
end

# see if propagating constants may be worthwhile
Expand Down
23 changes: 23 additions & 0 deletions test/compiler/inline.jl
Original file line number Diff line number Diff line change
Expand Up @@ -464,6 +464,29 @@ let
end
end

# force constant-prop' for `setproperty!`
let m = Module()
code = @eval m begin
# if we don't force constant-prop', `T = fieldtype(Foo, ::Symbol)` will be union-split to
# `Union{Type{Any},Type{Int}` and it will make `convert(T, nothing)` too costly
# and it leads to inlining failure
mutable struct Foo
val
_::Int
end

function setter(xs)
for x in xs
x.val = nothing
end
end

$code_typed1(setter, (Vector{Foo},))
end

@test !any(x->isinvoke(x, :setproperty!), code)
end

# validate inlining processing

@constprop :none @inline validate_unionsplit_inlining(@nospecialize(t)) = throw("invalid inlining processing detected")
Expand Down