Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support speculative execution on YARN for straggler task #55

Open
zhangpengshan opened this issue Oct 29, 2014 · 0 comments
Open

Support speculative execution on YARN for straggler task #55

zhangpengshan opened this issue Oct 29, 2014 · 0 comments

Comments

@zhangpengshan
Copy link
Contributor

For MapReduce version, we cannot enable speculative execution because we cannot control which task need to be enabled speculative execution. It is based on mapper progress while mapper progress is almost the same and once some task is proved to be faster, such speculative task attempt should be killed. It is hard in MapReduce.

In YARN, we have the freedom to do that, to build specific speculative mechanism.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant