Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redundant and deadlock-prone end listener removal #52

Merged
merged 2 commits into from
Aug 14, 2018
Merged

Conversation

gerashegalov
Copy link
Contributor

Related issues
Not yet

Describe the proposed solution
Skip removal of application end listener
a) because it's only called on spark.stop and won't ever be reused
b) because in Spark 2.3.1 it creates a deadlock between stop and onApplicationEnd

Describe alternatives you've considered

  • Removing listener after every run
  • stopping/recreating spark context beforeEach

Additional context
None

@codecov
Copy link

codecov bot commented Aug 14, 2018

Codecov Report

Merging #52 into master will decrease coverage by <.01%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master      #52      +/-   ##
==========================================
- Coverage   84.77%   84.77%   -0.01%     
==========================================
  Files         298      298              
  Lines        9752     9751       -1     
  Branches      353      548     +195     
==========================================
- Hits         8267     8266       -1     
  Misses       1485     1485
Impacted Files Coverage Δ
...ain/scala/com/salesforce/op/OpWorkflowRunner.scala 86.95% <ø> (-0.12%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d6f3d1b...b9948e0. Read the comment docs.

@tovbinm tovbinm merged commit 5805c97 into master Aug 14, 2018
@tovbinm tovbinm deleted the gera/fix-listener branch August 14, 2018 13:32
@tovbinm
Copy link
Collaborator

tovbinm commented Aug 14, 2018

How did you discover it? @gerashegalov

@gerashegalov
Copy link
Contributor Author

@tovbinm I took a thread dump and then looked into solutions.

@tovbinm
Copy link
Collaborator

tovbinm commented Aug 14, 2018

Thank you again @gerashegalov !!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants