Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

reset RandomDevice file from __init__ #42537

Merged
merged 1 commit into from
Oct 27, 2021
Merged

reset RandomDevice file from __init__ #42537

merged 1 commit into from
Oct 27, 2021

Conversation

JeffBezanson
Copy link
Sponsor Member

This prevents us from seeing an invalid IOStream object from a saved system image, and also ensures the files are initialized once for all threads.

We don't necessarily need both files, but eagerly initializing them both is much simpler than any alternative I can think of.

@JeffBezanson JeffBezanson added domain:randomness Random number generation and the Random stdlib kind:bugfix This change fixes an existing bug backport 1.6 Change should be backported to release-1.6 backport 1.7 labels Oct 7, 2021
KristofferC
KristofferC reviewed Oct 9, 2021
View reviewed changes
stdlib/Random/src/RNGs.jl Outdated
DEV_RANDOM[] = try
open("/dev/random")
catch
nothing
Copy link
Sponsor Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not set them unconditionally to nothing and let them be opened when they are needed?

Copy link
Sponsor Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would be a race condition. I guess it's ok if we're willing to leak nthreads file descriptors.

Copy link
Sponsor Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that's what the current code has as well, right? But might as well go with this, shouldn't really have a measurable startup cost.

Copy link
Sponsor Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can now declare it @atomic, and use replacefield!(nothing => fd), and then close it if you lost the race. Eventually we will likely want to add a Once-like object, which runs a thunk exactly once.

Copy link
Sponsor Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would be good; are we ok with not backporting to 1.6?

@KristofferC KristofferC mentioned this pull request Oct 11, 2021
Merged
84 tasks
@KristofferC KristofferC mentioned this pull request Oct 19, 2021
Merged
95 tasks
@JeffBezanson JeffBezanson changed the title cache RandomDevice file from __init__ reset RandomDevice file from __init__ Oct 20, 2021
@KristofferC KristofferC mentioned this pull request Oct 22, 2021
Merged
66 tasks
@JeffBezanson @KristofferC
reset RandomDevice file from __init__
98f538d 
This prevents us from seeing an invalid `IOStream` object from a saved
system image, and also ensures the files are opened once for all
threads.
@KristofferC
Copy link
Sponsor Member

Rebased to hopefully improve CI.

@JeffBezanson JeffBezanson removed the backport 1.6 Change should be backported to release-1.6 label Oct 27, 2021
@JeffBezanson JeffBezanson merged commit c74814e into master Oct 27, 2021
@JeffBezanson JeffBezanson deleted the jb/initrandomdevice branch October 27, 2021 20:34
aviatesk pushed a commit that referenced this pull request Oct 28, 2021
@JeffBezanson @aviatesk
reset RandomDevice file from __init__ (#42537)
9528183 
This prevents us from seeing an invalid `IOStream` object from a saved
system image, and also ensures the files are opened once for all
threads.
KristofferC pushed a commit that referenced this pull request Oct 28, 2021
@JeffBezanson @KristofferC
reset RandomDevice file from __init__ (#42537)
84b7ded 
This prevents us from seeing an invalid `IOStream` object from a saved
system image, and also ensures the files are opened once for all
threads.

(cherry picked from commit c74814e)
N5N3 added a commit to N5N3/julia that referenced this pull request Oct 29, 2021
@N5N3
Squashed commit of the following:
3a3b935 
commit c054dbc
Author: Shuhei Kadowaki <[email protected]>
Date:   Fri Oct 29 01:31:55 2021 +0900

    optimizer: eliminate allocations (JuliaLang#42833)

commit 6a9737d
Author: Jeff Bezanson <[email protected]>
Date:   Thu Oct 28 12:23:53 2021 -0400

    fix JuliaLang#42659, move `jl_coverage_visit_line` to runtime library (JuliaLang#42810)

commit c762f10
Author: Marc Ittel <[email protected]>
Date:   Thu Oct 28 12:19:13 2021 +0200

    change `julia` to `julia-repl` in docstrings (JuliaLang#42824)

    Co-authored-by: Michael Abbott <[email protected]>

commit 9f52ec0
Author: Dilum Aluthge <[email protected]>
Date:   Thu Oct 28 05:30:11 2021 -0400

    CI (Buildkite): Update all rootfs images to the latest versions (JuliaLang#42802)

    * CI (Buildkite): Update all rootfs images to the latest versions

    * Re-sign all of the signed pipelines

commit 404e584
Author: DilumAluthgeBot <[email protected]>
Date:   Wed Oct 27 21:11:04 2021 -0400

    🤖 Bump the Statistics stdlib from 74897fe to 5256d57 (JuliaLang#42826)

    Co-authored-by: Dilum Aluthge <[email protected]>

commit c74814e
Author: Jeff Bezanson <[email protected]>
Date:   Wed Oct 27 16:34:46 2021 -0400

    reset `RandomDevice` file from `__init__` (JuliaLang#42537)

    This prevents us from seeing an invalid `IOStream` object from a saved
    system image, and also ensures the files are opened once for all
    threads.

commit 05ed348
Author: Jeff Bezanson <[email protected]>
Date:   Wed Oct 27 15:24:17 2021 -0400

    only visit nonfunction_mt once when traversing method tables (JuliaLang#42821)

commit d71b77d
Author: DilumAluthgeBot <[email protected]>
Date:   Tue Oct 26 20:39:08 2021 -0400

    🤖 Bump the Downloads stdlib from 5f1509d to dbb0625 (JuliaLang#42811)

    Co-authored-by: Dilum Aluthge <[email protected]>

commit b4fddc1
Author: DilumAluthgeBot <[email protected]>
Date:   Tue Oct 26 14:46:20 2021 -0400

    🤖 Bump the Pkg stdlib from bc32103f to 26918395 (JuliaLang#42806)

    Co-authored-by: Dilum Aluthge <[email protected]>

commit 6a386de
Author: Dilum Aluthge <[email protected]>
Date:   Tue Oct 26 12:15:51 2021 -0400

    CI (Buildkite): make sure to hit ignore any unencrypted repo keys, regardless of where they are located in the repository (JuliaLang#42803)

commit 021a6b5
Author: Shuhei Kadowaki <[email protected]>
Date:   Wed Oct 27 01:08:33 2021 +0900

    optimizer: clean up inlining test code (JuliaLang#42804)

commit 16eb196
Merge: 21ebabf 1510eaa
Author: Shuhei Kadowaki <[email protected]>
Date:   Tue Oct 26 23:25:41 2021 +0900

    Merge pull request JuliaLang#42766 from JuliaLang/avi/42754

    optimizer: fix JuliaLang#42754, inline union-split const-prop'ed sources

commit 21ebabf
Author: Kristoffer Carlsson <[email protected]>
Date:   Tue Oct 26 16:11:32 2021 +0200

    simplify code loading test now that TOML files are parsed with a real TOML parser (JuliaLang#42328)

commit 1510eaa
Author: Shuhei Kadowaki <[email protected]>
Date:   Mon Oct 25 01:35:12 2021 +0900

    optimizer: fix JuliaLang#42754, inline union-split const-prop'ed sources

    This commit complements JuliaLang#39754 and JuliaLang#39305: implements a logic to use
    constant-prop'ed results for inlining at union-split callsite.
    Currently it works only for cases when constant-prop' succeeded for all
    (union-split) signatures.

    > example
    ```julia
    julia> mutable struct X
               # NOTE in order to confuse `fieldtype_tfunc`, we need to have at least two fields with different types
               a::Union{Nothing, Int}
               b::Symbol
           end;

    julia> code_typed((X, Union{Nothing,Int})) do x, a
               # this `setproperty` call would be union-split and constant-prop will happen for
               # each signature: inlining would fail if we don't use constant-prop'ed source
               # since the approximated inlining cost of `convert(fieldtype(X, sym), a)` would
               # end up very high if we don't propagate `sym::Const(:a)`
               x.a = a
               x
           end |> only |> first
    ```

    > before this commit
    ```julia
    CodeInfo(
    1 ─ %1 = Base.setproperty!::typeof(setproperty!)
    │   %2 = (isa)(a, Nothing)::Bool
    └──      goto #3 if not %2
    2 ─ %4 = π (a, Nothing)
    │        invoke %1(_2::X, 🅰️:Symbol, %4::Nothing)::Any
    └──      goto #6
    3 ─ %7 = (isa)(a, Int64)::Bool
    └──      goto #5 if not %7
    4 ─ %9 = π (a, Int64)
    │        invoke %1(_2::X, 🅰️:Symbol, %9::Int64)::Any
    └──      goto #6
    5 ─      Core.throw(ErrorException("fatal error in type inference (type bound)"))::Union{}
    └──      unreachable
    6 ┄      return x
    )
    ```

    > after this commit
    ```julia
    CodeInfo(
    1 ─ %1 = (isa)(a, Nothing)::Bool
    └──      goto #3 if not %1
    2 ─      Base.setfield!(x, :a, nothing)::Nothing
    └──      goto #6
    3 ─ %5 = (isa)(a, Int64)::Bool
    └──      goto #5 if not %5
    4 ─ %7 = π (a, Int64)
    │        Base.setfield!(x, :a, %7)::Int64
    └──      goto #6
    5 ─      Core.throw(ErrorException("fatal error in type inference (type bound)"))::Union{}
    └──      unreachable
    6 ┄      return x
    )
    ```

commit 4c3ae20
Author: Chris Foster <[email protected]>
Date:   Tue Oct 26 21:48:32 2021 +1000

    Make Base.ifelse a generic function (JuliaLang#37343)

    Allow user code to directly extend `Base.ifelse` rather than needing a
    special package for it.

commit 2e388e3
Author: Shuhei Kadowaki <[email protected]>
Date:   Mon Oct 25 01:30:09 2021 +0900

    optimizer: eliminate excessive specialization in inlining code

    This commit includes several code quality improvements in inlining code:
    - eliminate excessive specializations around:
      * `item::Pair{Any, Any}` constructions
      * iterations on `Vector{Pair{Any, Any}}`
    - replace `Pair{Any, Any}` with new, more explicit data type `InliningCase`
    - remove dead code
LilithHafner pushed a commit to LilithHafner/julia that referenced this pull request Feb 22, 2022
@JeffBezanson @LilithHafner
reset RandomDevice file from __init__ (JuliaLang#42537)
3000eb7 
This prevents us from seeing an invalid `IOStream` object from a saved
system image, and also ensures the files are opened once for all
threads.
LilithHafner pushed a commit to LilithHafner/julia that referenced this pull request Mar 8, 2022
@JeffBezanson @LilithHafner
reset RandomDevice file from __init__ (JuliaLang#42537)
8e3f296 
This prevents us from seeing an invalid `IOStream` object from a saved
system image, and also ensures the files are opened once for all
threads.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vtjnash vtjnash vtjnash left review comments

@KristofferC KristofferC KristofferC left review comments

Assignees
No one assigned
Labels
domain:randomness Random number generation and the Random stdlib kind:bugfix This change fixes an existing bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@JeffBezanson @KristofferC @vtjnash