Surprising and undocumented import handling in Distributed.@everywhere

[mwallerb]

Consider the following code:

    using Distributed
    addprocs(1)
    @everywhere begin
          # assume MyModule is defined in the path given
          push!(LOAD_PATH, "/path/to/my/module")
          import MyModule
    end

This fails with a LoadError, because @everywhere cherry-picks the imports for each everywhere block and runs them first regardless of the order of statements inside the block. Instead, one has to do this:

    # (...)
    @everywhere push!(LOAD_PATH, "/path/to/my/module")
    @everywhere import MyModule

This behaviour thoroughly breaks my internal parser, because these two blocks look equivalent (modulo an unnecessary synchronization point in the middle). Worse, the behaviour is not documented, and in fact contradicts the documentation.

Unfortunately, no rationale is given in the code as to why this was chosen. However, I think at the very least, one should amend the documentation with a corresponding note and add a rationale for this in the code.

One could maybe also issue a warning for @everywhere blocks which do not have import statements as their first statements. This would help poor newbies like me :)

Julia version: 1.2.0 (also present in master)

[vchuravy]

x-ref: JuliaLang/julia#21718

[joseortiz3]

Thank you!! I was getting very strange errors LoadError and UndefVarError in my worker processes, as if modules I loaded in

@everywhere begin
    import Random
    using Printf
    using GraphPlot
    using JuMP
    import CSV
    using Profile

    <other non-import-related code here>
end

were not actually loaded at all. Putting @everywhere at the beginning of just the import and using lines fixed it. All my other code (functions, variables, etc) was fine inside a single @everywhere begin .... end block. Thanks for pointing this out, hope others find this useful.