include() relative paths are different on worker processes

[mebrunet]

The behavior of @everywhere include seems inconsistent/broken. (I'm using Julia 0.6.3 on OSX)

Suppose you have a repository structure:

repo/
   src/
       -MyMod.jl
       -script.jl

with “MyMod.jl” containing:

module MyMod
    export f
    f(x) = 5 * x
end

and “script.jl” containing:

@everywhere include("MyMod.jl")
using MyMod

for p in workers()
    @show remotecall_fetch(f, p, p)
end

From within repo/src/ you can run julia -p 4 script.jl successfully, but from repo/ you cannot run Julia with multiple processes (i.e. call julia -p 4 src/script.jl), since the worker processes attempt to load repo/MyMod.jl which doesn’t exist.

This seems to be because include interprets relative paths differently when run on the main process than it does when run on worker processes. On main "relative" is relative to the script/file, whereas on workers it is relative to the current working directory.

(Note the behavior of include allows you to run "script.jl" on a single process from anywhere.)

As a workaround I've been using:

function include_everywhere(filepath)
    include(filepath) # Load on Node 1 first, triggering any precompile
    if nprocs() > 1
        fullpath = joinpath(@__DIR__, filepath)
        @sync for p in workers()
            @async remotecall_wait(include, p, fullpath)
        end
    end
end

include_everywhere("MyMod.jl")
using MyMod

But this feels like it ought to be the default behavior of @everywhere include.

[ViralBShah]

Also on discourse: https://discourse.julialang.org/t/everywhere-include-does-not-work-with-relative-file-paths/9598

[jonathan-laurent]

I am surprised that this did not receive more attention. This is literally the first thing I ran into when trying Distributed.
This is all the more surprising since the documentation is introducing @everywhere include(...) as a standard idiom.

[jonathan-laurent]

Also, the proposed fix will work when using several processes on the same machine but it will break when using multiple machines.