Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use actor based parallelization #266

Merged
merged 1 commit into from
Nov 30, 2014
Merged

Use actor based parallelization #266

merged 1 commit into from
Nov 30, 2014

Conversation

mbj
Copy link
Owner

@mbj mbj commented Oct 23, 2014

  • Fixes longstanding issues with nested parrallel execution. (good thing)
  • More responsive UI as progress reports happen in main thread with time shedule and not per mutation state event callback. (good thing)
  • Allows worker self registration (groundwork for distributed killing)
  • Performance is on-par with parallel. (good thing)
  • Removes frame throttle logic from reporters (good thing).
  • No need to make Mutant::Mutation instances serializable, reduces building partial result objects to make them transmittable over parallel's interfaces. (good thing)
  • More code. (bad thing)
  • 100% line coverage (I know I know, its not MT covered fully right now) but better than master ;)

@plexus
Copy link
Contributor

plexus commented Oct 23, 2014

Mutant configuration:
Matcher:            #<Mutant::Matcher::Config match_expressions=[<Mutant::Expression: Yaks*>] subject_ignores=[] subject_selects=[]>
Integration:        rspec
Expect Coverage:    100.00%
Jobs:               4
Includes:           ["lib"]
Requires:           ["yaks"]
Available Subjects: 150
Subjects:           150
Mutations:          5543
Kills:              21
Alive:              303
Runtime:            21.52s
Killtime:           1.48s
Overhead:           1351.55%
Coverage:           6.48%
Expected:           100.00%
Active subjects:    1
Yaks::CollectionMapper#call:/home/arne/github/yaks/yaks/lib/yaks/collection_mapper.rb:9 mutations: 90
- rspec:Yaks::CollectionMapper
...............
(15/90) 100% - killtime: 1.01s runtime: 5.67s overhead: 4.67s
/home/arne/.gem/ruby/2.1.3/gems/parallel-1.3.3/lib/parallel.rb:61:in `rescue in work': Parallel::DeadWorker (Parallel::DeadWorker)
        from /home/arne/.gem/ruby/2.1.3/gems/parallel-1.3.3/lib/parallel.rb:58:in `work'
        from /home/arne/.gem/ruby/2.1.3/gems/parallel-1.3.3/lib/parallel.rb:269:in `block (4 levels) in work_in_processes'
        from /home/arne/.gem/ruby/2.1.3/gems/parallel-1.3.3/lib/parallel.rb:439:in `with_instrumentation'
        from /home/arne/.gem/ruby/2.1.3/gems/parallel-1.3.3/lib/parallel.rb:268:in `block (3 levels) in work_in_processes'
        from /home/arne/.gem/ruby/2.1.3/gems/parallel-1.3.3/lib/parallel.rb:263:in `loop'
        from /home/arne/.gem/ruby/2.1.3/gems/parallel-1.3.3/lib/parallel.rb:263:in `block (2 levels) in work_in_processes'
        from /home/arne/.gem/ruby/2.1.3/gems/parallel-1.3.3/lib/parallel.rb:136:in `block (2 levels) in in_threads'

:(

@mbj
Copy link
Owner Author

mbj commented Oct 23, 2014

@plexus See it terminated ;)

@plexus
Copy link
Contributor

plexus commented Oct 23, 2014

\o/ lol

@mbj
Copy link
Owner Author

mbj commented Oct 23, 2014

@plexus My whole point was to make the error visible in the first step. I really hate systems that do not blow up. The complex interactions between the inner (killfork isolation) and the outer one (for parallelization) did hide this error before.

@mbj mbj force-pushed the actors branch 9 times, most recently from 3f60518 to 39ad443 Compare November 20, 2014 03:07
@mbj
Copy link
Owner Author

mbj commented Nov 20, 2014

@plexus The current state of this branch implements everything I wanted it to do. It includes the concurency semantics of master plus per worker reporting and some parallel related bugfixes (via removing it). If you wanted you could run it to give it some testing.

end # Message

# Undefined message payload
module Undefined
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does Undefined need to be a module? Would it be more lightweight to use Undefined = BasicObject.new or something?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dkubb You get a sane #inspect for free this way.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But I agree Module has too much public interface (I initially used Class). I'll use an anonymous object instance with an explicit #inspect defined.

@dkubb
Copy link
Collaborator

dkubb commented Nov 20, 2014

@mbj I finished a pass over the code.

@mbj mbj force-pushed the actors branch 3 times, most recently from 79f7a96 to ff7177a Compare November 23, 2014 16:13
@mbj mbj force-pushed the actors branch 2 times, most recently from 1020ff9 to a29d799 Compare November 30, 2014 23:07
@mbj mbj changed the title [WIP] Use actor based parallelization Use actor based parallelization Nov 30, 2014
mbj added a commit that referenced this pull request Nov 30, 2014
Use actor based parallelization
@mbj mbj merged commit dea853d into master Nov 30, 2014
@mbj mbj deleted the actors branch November 30, 2014 23:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants