Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backports for 1.7.3 #44189

Merged
merged 40 commits into from
Apr 22, 2022
Merged

Backports for 1.7.3 #44189

merged 40 commits into from
Apr 22, 2022

Conversation

KristofferC
Copy link
Sponsor Member

@KristofferC KristofferC commented Feb 15, 2022

Backported PRs:

Need manual backport:

Contains multiple commits, manual intervention needed:

Non-merged PRs with backport label:

bkamins and others added 6 commits February 15, 2022 14:48
We could also consider adding `defalg(v::AbstractArray{<:Number}) = DEFAULT_UNSTABLE`,
but it is unlikely that someone will want to do `Union` of `<:Number`
and something other than `Missing` that still would support comparison.

Relevant for JuliaStrings/InlineStrings.jl#21
(and other custom types that will want to add `defalg` support allowing
for `Union` with `Missing`).

(cherry picked from commit eb724e0)
(cherry picked from commit 4c101a1)
@KristofferC KristofferC added the kind:release Release management and versioning. label Feb 15, 2022
@KristofferC KristofferC changed the title Backports release 1.7 Backports for 1.7.3 Feb 15, 2022
KristofferC and others added 10 commits February 23, 2022 08:28
* fix UndefRefError for documentation system

* add a test

Co-authored-by: Jameson Nash <[email protected]>
(cherry picked from commit 4842007)
…ltiplication (#44219)

Co-authored-by: Fredrik Ekre <[email protected]>
(cherry picked from commit 928f63c)
On Windows, we observed occasional issues where an error within the
function callback to the `open(::Function, ::Cmd)` method would cause
problems due to assuming that the opened process had finished by the
time the `open()` call was finished.  In most cases this was true,
however on Windows, it was found that we need to explicitly `wait()`
upon the process object to ensure that all file handles held by the
subprocess were properly closed by the time `open()` is finished.

Co-authored-by: Dilum Aluthge <[email protected]>
(cherry picked from commit 623ceb7)
* [CPUID] Rework how current ISA is determined

* [CPUID] Add ISA entry for A64FX

* [CPUID] Add ISA entry for Apple Silicon M1

* [CPUID] Simplify collection of full set of features for architecture

* [CPUID] Remove AES from A64FX ISA, not all chips appear to have it

(cherry picked from commit f45b6ad)
Co-authored-by: Jameson Nash <[email protected]>
Co-authored-by: Fredrik Ekre <[email protected]>
(cherry picked from commit f5d9b86)
@sostock
Copy link
Contributor

sostock commented Feb 23, 2022

#41513 and #42054 should be backported as well.

Because we might not have synchronized the list again yet.

(cherry picked from commit cac90b5)
We have 2 copies of this data, and so need to make sure we are pointing
at the correct one for runtime.

(cherry picked from commit 2f1f2f6)
ericphanson and others added 7 commits March 6, 2022 08:17
Because we're starting to distribute macOS tarballs as well, let's
codesign them by default, when possible.

(cherry picked from commit 6b29ebd)
…et (#44262)

We collect the relocations (i.e. the GOT slots that is used in the code) for each target
in `tgt.relocs`. Needing a relocation, however, does not imply that the function is cloned
for this target within the group (It does mean that at least one target
in the group has it cloned). The previous version would miss the relocation in this case.

This was triggerred with the following cloning situation

    caller: clone_1
    callee: clone_1, clone_1.clone_3

Since caller.clone_1 may call either callee.clone_1 or callee.clone_1.clone_3 a relocation
for callee will be used and is required to be initialized.
In addition to target 1, target 2 (and in fact target 3) within group 1
will also use caller.clone_1. However, since callee isn't cloned for target 2
the previous version wouldn't have saved this slot in the relocation array.

(cherry picked from commit 76fc067)
Co-authored-by: Simeon David Schaub <[email protected]>
(cherry picked from commit 2338f5d)
staticfloat and others added 10 commits April 19, 2022 13:10
If a user runs the tests with a `~/.gitconfig` that provides an
`init.defaultBranch` that is not `master`, our tests fail.  Let's adjust
to the user's configuration as appropriate.  We'll also rename this to
`default_branch` to signify that it may not be called `master` anymore.

(cherry picked from commit ac1d693)
It also fixes `round(Integer, big(NaN))`.

Solves #44662

(cherry picked from commit ecf3558)
Discovered while running the syntax tests with lines 25-28 of
`jlfrontend.scm` commented out. Turned out we didn't handle `.op`
correctly in neither `check-dotop` nor in `deparse`. This meant we just
got `error: malformed expression` instead of an actually useful error
message.

(cherry picked from commit 9112135)
We were accidentally passing the start of the list instead of the end of
the list, resulting in some values passing through uninitialized.

Fix #42297 regression

(cherry picked from commit 5f2abf6)
…#44972)

* [REPL] remove erroneous space in test
  Introduced by #33805 (74f2de1).
* Revert "Temporarily move the `REPL` test suite to node 1, to buy us time until we fix the underlying bugs (#44961)"
  This reverts commit 322fd70.

(cherry picked from commit fbec395)
(cherry picked from commit c589e0d)
Note: this is the first build of the real upstream version 1.2.12 which was
released a few days ago.

(cherry picked from commit 81e7cfc)
The event might have triggered on another thread before we observed it
here, or it might have gotten finalized before it got triggered. Either
outcome can result in a lost event. (I observed the later situation
occurring locally during the Dates test once).

(cherry picked from commit 8cc00ff)
fixes #44723

Co-authored-by: Takafumi Arakaki <[email protected]>
(cherry picked from commit 19eb307)
@KristofferC
Copy link
Sponsor Member Author

@nanosoldier runtests(ALL, vs = ":release-1.7")

@KristofferC
Copy link
Sponsor Member Author

@nanosoldier runbenchmarks(!"scalar", vs=":release-1.7")

@nanosoldier
Copy link
Collaborator

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here.

@nanosoldier
Copy link
Collaborator

Your package evaluation job has completed - possible new issues were detected. A full report can be found here.

@KristofferC

This comment was marked as outdated.

@giordano
Copy link
Contributor

Can #44671 also be included here? 🙂

* avoid using `@sync_add` on remotecalls

It seems like @sync_add adds the Futures to a queue (Channel) for @sync, which
in turn calls wait() for all the futures synchronously. Not only that is
slightly detrimental for network operations (latencies add up), but in case of
Distributed the call to wait() may actually cause some compilation on remote
processes, which is also wait()ed for. In result, some operations took a great
amount of "serial" processing time if executed on many workers at once.

For me, this closes #44645.

The major change can be illustrated as follows: First add some workers:

```
using Distributed
addprocs(10)
```

and then trigger something that, for example, causes package imports on the
workers:

```
using SomeTinyPackage
```

In my case (importing UnicodePlots on 10 workers), this improves the loading
time over 10 workers from ~11s to ~5.5s.

This is a far bigger issue when worker count gets high. The time of the
processing on each worker is usually around 0.3s, so triggering this problem
even on a relatively small cluster (64 workers) causes a really annoying delay,
and running `@everywhere` for the first time on reasonable clusters (I tested
with 1024 workers, see #44645) usually takes more than 5 minutes. Which sucks.

Anyway, on 64 workers this reduces the "first import" time from ~30s to ~6s,
and on 1024 workers this seems to reduce the time from over 5 minutes (I didn't
bother to measure that precisely now, sorry) to ~11s.

Related issues:
- Probably fixes #39291.
- #42156 is a kinda complementary -- it removes the most painful source of
  slowness (the 0.3s precompilation on the workers), but the fact that the
  wait()ing is serial remains a problem if the network latencies are high.

May help with #38931

Co-authored-by: Valentin Churavy <[email protected]>
(cherry picked from commit 62e0729)
@KristofferC
Copy link
Sponsor Member Author

@nanosoldier runtests(["ADNLPModels", "AssociativeArrays", "BestApproximation", "CalibrateEDMF", "Causal", "CrystalInfoFramework", "ExponentialUtilities", "FlashWeave", "Groebner", "HomotopyContinuation", "ITensorGaussianMPS", "ITensorVisualizationBase", "Infernal", "IntensityScans", "IntervalTrees", "KeplerTools", "MPSToolkit", "NumericalAlgorithms", "OrbitalTrajectories", "PProf", "Pfam", "Probably", "ProgressiveHedging", "QXTns", "QXTools", "RuntimeGeneratedFunctions", "SLEEFPirates", "SimpleBufferStream", "TensorNetworkCodes", "YaoQX"], vs = ":release-1.7")

@nanosoldier
Copy link
Collaborator

Your package evaluation job has completed - possible new issues were detected. A full report can be found here.

@KristofferC
Copy link
Sponsor Member Author

Failure for BestApproximation is a bugfix from #44676 which makes the package test fail.

@KristofferC
Copy link
Sponsor Member Author

@nanosoldier runtests(["AssociativeArrays", "BestApproximation", "HomotopyContinuation", "ITensorGaussianMPS", "ITensorVisualizationBase", "MPSToolkit", "NumericalAlgorithms", "QXTns", "QXTools", "TensorNetworkCodes", "YaoQX"], vs = ":release-1.7")

@nanosoldier
Copy link
Collaborator

Your package evaluation job has completed - possible new issues were detected. A full report can be found here.

@DilumAluthge
Copy link
Member

@staticfloat Will we need to turn on all of the buildbots for Kristoffer and Alex to be able to get binaries for this release?

@staticfloat
Copy link
Sponsor Member

Yes, I believe so.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind:release Release management and versioning.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet