Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC: add procs argument to @-everywhere and move code to a function #22589

Merged
merged 1 commit into from
Jul 24, 2017

Conversation

vtjnash
Copy link
Sponsor Member

@vtjnash vtjnash commented Jun 28, 2017

Adding a function remotecall_eval lets us simplify code that previously needed awkward contortions to use remotecall_fetch on an anonymous function. This allows us to transform the more complex pattern of:

@sync for pid in pids
    @async remotecall_fetch(Core.eval, wid, Main, quote
        expression
    end)
end

into:

@everywhere pids expression

which resolves #18166

(note: This is against PR #22588, since I wanted to use this functionality to simplify its tests. When the RFCs are removed, I'll rebase whichever ones we decide to merge first.)

@kshyatt kshyatt added the domain:parallelism Parallel or distributed computation label Jun 28, 2017
@kshyatt kshyatt requested a review from amitmurthy June 28, 2017 01:50
`@everywhere` with [`@eval`](@ref) allows us to broadcast
local variables using interpolation :
`@everywhere` does not capture any local variables.
Instead, local variable can be broadcast using interpolation:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

variables

yield() # ensure that the remotecall_fetch has been started
remotecall_eval(m::Module, procs, expression)

Execute an expression under module `m` on all procs.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

on the specified procs

@tkelman
Copy link
Contributor

tkelman commented Jun 28, 2017

this by itself is less controversial than #22588. please decouple them.

@andreasnoack andreasnoack force-pushed the jn/no-node1-include branch 2 times, most recently from a592f3f to f8b84f2 Compare July 17, 2017 02:17
@tkelman tkelman changed the base branch from jn/no-node1-include to master July 17, 2017 13:20
unshift!(LOAD_PATH, $load_path)
unshift!(LOAD_CACHE_PATH, $load_cache_path)
end)
@everywhere test_workers begin
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a pretty indirect test, could use something more specific in the distributed tests

adding remotecall_eval lets us simplify code that previously needed
awkward contortions to use remotecall_fetch on an anonymous function

fixes a code-duplication bug in the case where something is spliced into expr

and removes and corrects misleading information in the doc string
@vtjnash vtjnash merged commit cf8a697 into master Jul 24, 2017
@vtjnash vtjnash deleted the jn/remotecall_eval branch July 24, 2017 14:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
domain:parallelism Parallel or distributed computation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Error invoking worker from module
3 participants