Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(controller): Same workflow nodes are not executing parallel even semaphore locks available #6418

Merged
merged 4 commits into from
Jul 27, 2021

Conversation

sarabala1979
Copy link
Member

closes #6110

Signed-off-by: Saravanan Balasubramanian [email protected]

Checklist:

Tips:

  • Your PR needs to pass the required checks before it can be approved. If the check is not required (e.g. E2E tests) it does not need to pass
  • Sign-off your commits to pass the DCO check: git commit --signoff.
  • Run make pre-commit -B to fix codegen or lint problems.
  • Say how how you tested your changes. If you changed the UI, attach screenshots.

@codecov
Copy link

codecov bot commented Jul 24, 2021

Codecov Report

Merging #6418 (7aaeb73) into master (4da8fd9) will decrease coverage by 0.13%.
The diff coverage is 23.45%.

❗ Current head 7aaeb73 differs from pull request most recent head 285eca4. Consider uploading reports for the commit 285eca4 to get more accurate results
Impacted file tree graph

@@            Coverage Diff             @@
##           master    #6418      +/-   ##
==========================================
- Coverage   48.81%   48.67%   -0.14%     
==========================================
  Files         253      253              
  Lines       18153    18223      +70     
==========================================
+ Hits         8861     8870       +9     
- Misses       8324     8385      +61     
  Partials      968      968              
Impacted Files Coverage Δ
config/config.go 31.57% <ø> (ø)
config/node_events.go 100.00% <ø> (ø)
util/kubeconfig/kubeconfig.go 19.51% <0.00%> (-5.69%) ⬇️
workflow/artifacts/gcs/gcs.go 8.51% <0.00%> (-0.64%) ⬇️
workflow/executor/emissary/emissary.go 0.00% <0.00%> (ø)
cmd/argoexec/commands/emissary.go 50.35% <40.00%> (+0.35%) ⬆️
workflow/sync/semaphore.go 92.59% <71.42%> (-1.53%) ⬇️
workflow/controller/operator.go 72.54% <85.71%> (+0.06%) ⬆️
workflow/metrics/server.go 15.78% <0.00%> (-3.51%) ⬇️
server/workflow/workflow_server.go 44.41% <0.00%> (-2.40%) ⬇️
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4da8fd9...285eca4. Read the comment docs.

Signed-off-by: Saravanan Balasubramanian <[email protected]>
@@ -165,7 +176,8 @@ func (s *PrioritySemaphore) tryAcquire(holderKey string) (bool, string) {
if s.pending.Len() > 0 {
item := s.pending.peek()
nextKey = fmt.Sprintf("%v", item.key)
if holderKey != nextKey {
if holderKey != nextKey && !isSameWorkflowNodeKeys(holderKey, nextKey) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't want to prioritize/order the same workflow nodes. The executetemplate will make sure the template order.

if the workflow has multiple nodes to wait for lock, the operator node order and semaphore queue order are going out of sync. if it same workflow key

@sarabala1979 sarabala1979 marked this pull request as ready for review July 26, 2021 16:13
@sarabala1979 sarabala1979 requested a review from alexec July 26, 2021 19:17
Copy link
Contributor

@alexec alexec left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please clean-up PR

Signed-off-by: Saravanan Balasubramanian <[email protected]>
@sarabala1979
Copy link
Member Author

please clean-up PR

done. I missed to cleanup my env

@sarabala1979 sarabala1979 merged commit 7ec5b3e into argoproj:master Jul 27, 2021
@sarabala1979 sarabala1979 mentioned this pull request Jul 27, 2021
39 tasks
sarabala1979 added a commit that referenced this pull request Jul 27, 2021
…semaphore locks available (#6418)

* fix(controller): Same workflow nodes are not executing parallel in semaphore
@sarabala1979 sarabala1979 mentioned this pull request Aug 2, 2021
49 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

semaphore lost
3 participants