Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Backports for Julia 1.3-RC3 #33221

Merged
merged 19 commits into from
Oct 3, 2019
Merged

Backports for Julia 1.3-RC3 #33221

merged 19 commits into from
Oct 3, 2019

Conversation

KristofferC
Copy link
Sponsor Member

@KristofferC KristofferC commented Sep 11, 2019 •
edited
Loading

Backported PRs:

@KristofferC KristofferC added needs nanosoldier run This PR should have benchmarks run on it needs pkgeval Tests for all registered packages should be run with this change kind:release Release management and versioning. labels Sep 11, 2019
@KristofferC
Copy link
Sponsor Member Author

@nanosoldier runbenchmarks(ALL, vs = ":release-1.2")

@nanosoldier
Copy link
Collaborator

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here. cc @ararslan

@KristofferC
Copy link
Sponsor Member Author

I'm getting this on PkgEval when building this:

[07:05:50] /workspace/srcdir/julia/src/array.c: In function 'jl_array_grow_at':
[07:05:50] /workspace/srcdir/julia/src/array.c:818:15: warning: 'newtypetagdata' may be used uninitialized in this function [-Wmaybe-uninitialized]
[07:05:50]          memset(newtypetagdata + idx, 0, inc);
[07:05:50]                ^
[07:05:50] /workspace/srcdir/julia/src/array.c:734:11: note: 'newtypetagdata' was declared here
[07:05:50]      char *newtypetagdata;
[07:05:50]            ^
[07:05:50] /workspace/srcdir/julia/src/array.c:799:37: warning: 'typetagdata' may be used uninitialized in this function [-Wmaybe-uninitialized]
[07:05:50]              if (isbitsunion) memmove(newtypetagdata + idx + inc, typetagdata + idx, n - idx);
[07:05:50]                                      ^
[07:05:50] /workspace/srcdir/julia/src/array.c:733:11: note: 'typetagdata' was declared here
[07:05:50]      char *typetagdata;
[07:05:50]            ^
[07:05:50]     CC src/llvm-final-gc-lowering.o
[07:05:50]     CC src/llvm-pass-helpers.o
[07:05:50]     CC src/llvm-late-gc-lowering.o
[07:05:51]     CC src/llvm-lower-handlers.o
[07:05:51]     CC src/llvm-gc-invariant-verifier.o
[07:05:51] /workspace/srcdir/julia/src/runtime_ccall.cpp: In function 'jl_value_t* jl_get_cfunction_trampoline(jl_value_t*, jl_datatype_t*, htable_t*, jl_svec_t*, void* (*)(void*, void**), jl_unionall_t*, jl_value_t**)':
[07:05:51] /workspace/srcdir/julia/src/runtime_ccall.cpp:309:29: warning: ISO C++ forbids casting between pointer-to-function and pointer-to-object [-Wpedantic]
[07:05:51]                      (void*)&trampoline_deleter
[07:05:51]                              ^
[07:05:51]     CC src/llvm-propagate-addrspaces.o
[07:05:51]     CC src/llvm-multiversioning.o
[07:05:51]     CC src/llvm-alloc-opt.o
[07:05:51]     CC src/cgmemmgr.o
[07:05:51]     CC src/llvm-api.o
[07:05:56] /workspace/srcdir/julia/src/codegen.cpp: In function 'void init_julia_llvm_env(llvm::Module*)':
[07:05:56] /workspace/srcdir/julia/src/codegen.cpp:7504:46: warning: ISO C++ forbids casting between pointer-to-function and pointer-to-object [-Wpedantic]
[07:05:56]      add_named_global(jl_newbits_func, (void*)jl_new_bits);
[07:05:56]                                               ^
[07:06:05] /workspace/srcdir/julia/src/codegen.cpp: In function 'jl_value_t* _ZL11static_evalR12jl_codectx_tP11_jl_value_tii.constprop.1272(jl_codectx_t&, jl_value_t*, int)':
[07:06:05] /workspace/srcdir/julia/src/codegen.cpp:1972:28: warning: variable 'n' might be clobbered by 'longjmp' or 'vfork' [-Wclobbered]
[07:06:05]                      size_t n = jl_array_dim0(e->args)-1;

anything that should be done about that?

tkf and others added 5 commits September 19, 2019 11:44
@tkf @KristofferC
Fix 5-arg mul! for tiled generic case (#33218)
dc642ad 
(cherry picked from commit 09ff912)
@andreasnoack @KristofferC
Define / method for adjoint LU rhs. (#33209)
cd2d247 
Fixes #33177

(cherry picked from commit 4c8f8fa)
@nickrobinson251 @KristofferC
Remove out-of-date docs on printing nothing (#33216)
5452037 
(cherry picked from commit 24ddae9)
@tkf @KristofferC
Dispatch more cases to BLAS.gemm! (#33229)
c780b9c 
* Dispatch more cases to BLAS.gemm!

* Use α and β instead of alpha′ and beta′

(cherry picked from commit 51b3227)
@staticfloat @KristofferC
macOS codesigning: only attempt if $MACOS_CODESIGN_IDENTITY is set (#…
adfae3c 
…33228)

This has the following advantages:

* It allows our buildbots to pass in which code signing identity they
want to use (rather than having the identity hash hardcoded in our build
system)

* It stops blindly attempting to codesign on random user's machines

* It causes codesign failure to stop the build

(cherry picked from commit 0760842)
@KristofferC
Copy link
Sponsor Member Author

KristofferC commented Sep 19, 2019 •
edited
Loading

PkgEval: https://gist.github.com/KristofferC/47577ec5ad3b4418fd98f20bc3d91957

@staticfloat
Copy link
Sponsor Member

I would like to selfishly request a Pkg bump to whatever the latest 1.3 branch is for Pkg, so that it contains JuliaLang/Pkg.jl@45be6f6, as that will be important for users of 1.3

@KristofferC
Copy link
Sponsor Member Author

It should be backported into JuliaLang/Pkg.jl#1361. That PR merged and then we bump the Pkg version to that commit on Julia master and then backport that commit when merged.

@ericphanson
Copy link
Contributor

I fixed the inference hang on TravelingSalesmanExact by typing a vector concretely: ericphanson/TravelingSalesmanExact.jl#7 (comment).

@KristofferC
Copy link
Sponsor Member Author

That's weird but thanks for fixing!

vtjnash and others added 5 commits September 22, 2019 16:55
@vtjnash
malloc wrappers: ensure thread-safe (#33284)
782715b 
Better align the API of the jl_ wrappers for malloc/realloc/free with the libc namesakes,
including being safe to use on threads.

fix #33223

(cherry picked from commit 6c2c940)
@JeffBezanson
fix #33270; stack overflow in named tuple ctor with Type{T} (#33303)
7861a35 
(cherry picked from commit 0cd4f09)
@JeffBezanson
fix #33227, lowering of empty goto loop at top level (#33230)
c494d50 
(cherry picked from commit 5fe17cd)
@staticfloat
Take a step toward complete BB domination (#33125)
3dd7002 
This provides a number of benefits:

* Significantly reduces the amount of work in win-extras, which I've
always thought was an annoying wart in our build system.

* Provides a consistent 7z executable on all platforms, which can be
picked up by Pkg/BinaryProvider for consistent
unzipping/untarring/ungzipping on all platforms. Note that on Windows,
7z is a repackaged binary from sourceforge and is the "full" 7z, whereas
on all other platforms it is a p7zip build of the standalone 7za
executable with all relevant patches applied. There are some format
capability differences, but for the important things it should be fine.

(cherry picked from commit b6ddd87)
@JeffBezanson @KristofferC
add a subtyping fast path for tuple of Unions <: Vararg
2c757e7 
fixes #33337

(cherry picked from commit 9dd7b5b)
@KristofferC
Copy link
Sponsor Member Author

KristofferC commented Sep 22, 2019 •
edited
Loading

I preemptively pushed #33353 to get an up to date PkgEval run.

@KristofferC
Copy link
Sponsor Member Author

PkgEval run didn't reveal anything interesting.

With Pkg bumped, I believe we should be able to tag RC-3 which actually has a chance to promote to 1.3.

@KristofferC
Copy link
Sponsor Member Author

Win32 seems borken: cp: cannot stat 'dist-extras/7z.*': No such file or directory

@staticfloat
Copy link
Sponsor Member

Win32 seems borken: cp: cannot stat 'dist-extras/7z.*': No such file or directory

That's an expected error; it's a temporary workaround for building Julia 1.2-, which didn't ship with 7z. That's why it's allowed to fail and not bother anything.

The real error is the signing error below, it looks like mono is broken. I'll take a look at the builder, in the meantime, I'm restarting the job.

@KristofferC
Copy link
Sponsor Member Author

Should look at #33370 before tagging RC3.

@KristofferC KristofferC added status:DO NOT MERGE Do not merge this PR! and removed needs nanosoldier run This PR should have benchmarks run on it labels Sep 24, 2019
@KristofferC KristofferC removed the needs pkgeval Tests for all registered packages should be run with this change label Sep 24, 2019
@RalphAS
Copy link

RalphAS commented Sep 28, 2019

Should #32327 be included? It's on the other backports lists.

StefanKarpinski and others added 8 commits October 1, 2019 14:43
@StefanKarpinski @KristofferC
JULIA_COPY_STACKS: validate values (#33318)
f0cb955 
(cherry picked from commit 9079472)
@yhls @KristofferC
fix bug related to block renaming for DCE
631989c 
(cherry picked from commit 55759af)
@ararslan @KristofferC
Fix source build of p7zip on FreeBSD (#33431)
67d0a51 
(cherry picked from commit 4fa6a62)
@KristofferC
Revert "Disable BB GMP and MPFR, to fix performance regressions"
b62bb3a 
This reverts commit d7187e4.
@KristofferC
fix not messing up current_taks().storage[:SOURCE_PATH] by allowing…
3bc74e0 
… the finally in include to run (#33447)

(cherry picked from commit 16a09b4)
@JeffBezanson @KristofferC
better fix for #33337; revert #33353 (#33440)
68222cc 
(cherry picked from commit d911a9f)
@staticfloat @KristofferC
Update GMP and MPFR builds to use better compiler flags (#33096)
889d7e7 
(cherry picked from commit b685cf6)
@JeffBezanson @KristofferC
fix #33370, avoid collisions between gensyms and anon function names (#…
1b4b7ef 
…33426)

(cherry picked from commit 888d32a)
@KristofferC KristofferC removed the status:DO NOT MERGE Do not merge this PR! label Oct 3, 2019
@KristofferC KristofferC changed the title WIP: Backports for Julia 1.3-RC3 Backports for Julia 1.3-RC3 Oct 3, 2019
@KristofferC
Copy link
Sponsor Member Author

@nanosoldier runbenchmarks(ALL, vs = ":release-1.2")

@KristofferC
Copy link
Sponsor Member Author

This should be good to go when CI + nanosoldier comes back ok.

@nanosoldier
Copy link
Collaborator

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here. cc @ararslan

@KristofferC
Copy link
Sponsor Member Author 

      From worker 6:	LLVM ERROR: out of memory
      From worker 6:	
      From worker 6:	This application has requested the Runtime to terminate it in an unusual way.
      From worker 6:	Please contact the application's support team for more information.
      From worker 6:	
      From worker 6:	signal (22): SIGABRT
      From worker 6:	in expression starting at C:\cygwin\home\Administrator\buildbot\worker-tabularasa\tester_win32\build\share\julia\stdlib\v1.3\Statistics\test\runtests.jl:136
      From worker 6:	raise at C:\Windows\SYSTEM32\msvcrt.dll (unknown line)
      From worker 6:	abort at C:\Windows\SYSTEM32\msvcrt.dll (unknown line)
      From worker 9:	Internal error: encountered unexpected error in runtime:
      From worker 9:	reset_page at /home/Administrator/buildbot/worker/package_win32/build/src/home/Administrator/buildbot/worker/package_win32/build/src\gc.c:1044 [inlined]
      From worker 9:	add_page at /home/Administrator/buildbot/worker/package_win32/build/src/home/Administrator/buildbot/worker/package_win32/build/src\gc.c:1079
      From worker 9:	jl_gc_pool_alloc at /home/Administrator/buildbot/worker/package_win32/build/src/home/Administrator/buildbot/worker/package_win32/build/src\gc.c:1132
      From worker 9:	jl_gc_alloc_ at /home/Administrator/buildbot/worker/package_win32/build/src/home/Administrator/buildbot/worker/package_win32/build/src\julia_internal.h:233 [inlined]
      From worker 9:	jl_gc_alloc at /home/Administrator/buildbot/worker/package_win32/build/src/home/Administrator/buildbot/worker/package_win32/build/src\gc.c:2940
      From worker 9:	_new_array_ at /home/Administrator/buildbot/worker/package_win32/build/src/home/Administrator/buildbot/worker/package_win32/build/src\array.c:100
      From worker 9:	_new_array at /home/Administrator/buildbot/worker/package_win32/build/src/home/Administrator/buildbot/worker/package_win32/build/src\array.c:160 [inlined]
      From worker 9:	jl_alloc_array_1d at /home/Administrator/buildbot/worker/package_win32/build/src/home/Administrator/buildbot/worker/package_win32/build/src\array.c:421
      From worker 9:	Array at .\boot.jl:404 [inlined]
      From worker 9:	Array at .\boot.jl:413 [inlined]
      From worker 9:	zeros at .\array.jl:459 [inlined]
      From worker 9:	zeros at .\array.jl:455 [inlined]
      From worker 9:	BitSet at .\bitset.jl:18 [inlined]
      From worker 9:	construct_ssa! at .\compiler/ssair\slot2ssa.jl:661
      From worker 9:	just_construct_ssa at .\compiler/ssair\driver.jl:107
      From worker 9:	run_passes at .\compiler/ssair\driver.jl:112
      From worker 9:	optimize at .\compiler\optimize.jl:169
      From worker 9:	typeinf at .\compiler\typeinfer.jl:33
      From worker 9:	abstract_call_method_with_const_args at .\compiler\abstractinterpretation.jl:243
      From worker 9:	abstract_call_gf_by_type at .\compiler\abstractinterpretation.jl:116

Lame :/. Could one of the recent commits have caused that?

@ararslan
Copy link
Member

ararslan commented Oct 3, 2019

CI looks okay now and skimming over the Nanosoldier regressions, they look like noise to me.

@staticfloat staticfloat merged commit 11561e2 into release-1.3 Oct 3, 2019
@staticfloat staticfloat deleted the backports-release-1.3 branch October 3, 2019 19:24
@musm musm mentioned this pull request Nov 16, 2019
Closed
vchuravy pushed a commit to JuliaLang/LazyArtifacts.jl that referenced this pull request Oct 2, 2023
@staticfloat
Backports for Julia 1.3-RC3 (JuliaLang/julia#33221)
db9eaa4 
Backports for Julia 1.3-RC3
vchuravy pushed a commit to JuliaLang/Distributed.jl that referenced this pull request Oct 6, 2023
@staticfloat
Backports for Julia 1.3-RC3 (JuliaLang/julia#33221)
e1b3e9c 
Backports for Julia 1.3-RC3
vchuravy pushed a commit to JuliaLang/Test.jl that referenced this pull request Oct 7, 2023
@staticfloat
Backports for Julia 1.3-RC3 (JuliaLang/julia#33221)
a7a6ad1 
Backports for Julia 1.3-RC3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ararslan ararslan ararslan approved these changes

Assignees
No one assigned
Labels
kind:release Release management and versioning.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet