Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Fix automatic partition hang when open timeout is too long exhausted automatic partition rpc thread pool #47838

Merged
merged 2 commits into from
Jul 10, 2024
Merged

[BugFix] Fix automatic partition hang when open timeout is too long exhausted automatic partition rpc thread pool #47838

merged 2 commits into from
Jul 10, 2024

Conversation

meegoo
Copy link
Contributor

@meegoo meegoo commented Jul 4, 2024 •
edited by kevincai
Loading

Why I'm doing:

Currently, the open operation uses half of the total ingestion timeout as the RPC timeout, which in exceptional cases may lead to the depletion of the automatic partition’s thread pool. Therefore, on one hand, the thread pool for automatic partition needs to be increased, and on the other hand, the maximum timeout for the open operation needs to be controlled.

What I'm doing:

Fixes #issue

What type of PR is this:

  • BugFix
  • Feature
  • Enhancement
  • Refactor
  • UT
  • Doc
  • Tool

Does this PR entail a change in behavior?

  • Yes, this PR will result in a change in behavior.
  • No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • Parameter changes: default values, similar parameters but with different default values
  • Policy changes: use new policy to replace old one, functionality automatically enabled
  • Feature removed
  • Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • I have added test cases for my bug fix or my new feature
  • This pr needs user documentation (for new or modified features or behaviors)
    • I have added documentation for my new feature or new function
  • This is a backport pr

Bugfix cherry-pick branch check:

  • I have checked the version labels which the pr will be auto-backported to the target branch
    • 3.3
    • 3.2
    • 3.1
    • 3.0
    • 2.5

@meegoo meegoo requested review from a team as code owners July 4, 2024 06:23
@github-actions github-actions bot added the documentation Improvements or additions to documentation label Jul 4, 2024
@mergify mergify bot assigned meegoo Jul 4, 2024
@meegoo meegoo enabled auto-merge (squash) July 8, 2024 06:03
@meegoo
[BugFix] Fix automatic partition hang when open timeout is too long e…
1b5a7a8 
…xhausted automatic partition rpc thread pool

Signed-off-by: meegoo <[email protected]>
jaogoy
jaogoy previously approved these changes Jul 9, 2024
View reviewed changes
wyb
wyb previously approved these changes Jul 9, 2024
View reviewed changes
@meegoo meegoo dismissed stale reviews from wyb and jaogoy via 0ad0c43 July 9, 2024 15:25
@meegoo meegoo disabled auto-merge July 9, 2024 15:25
@kevincai
Copy link
Contributor

kevincai commented Jul 9, 2024

mark this PR as behavior changed in two places

  1. default value change for the tablet write timeout
  2. number of max threads changed for the auto partition.

@github-actions GitHub Actions
Copy link

github-actions bot commented Jul 9, 2024

[FE Incremental Coverage Report]

✅ pass : 0 / 0 (0%)

@github-actions GitHub Actions
Copy link

github-actions bot commented Jul 9, 2024

[BE Incremental Coverage Report]

✅ pass : 2 / 2 (100.00%)

file detail

path covered_line new_line coverage not_covered_line_detail
🔵 be/src/exec/tablet_sink_index_channel.cpp 2 2 100.00% []

@meegoo meegoo merged commit 3fadc53 into StarRocks:main Jul 10, 2024
57 of 59 checks passed
@github-actions GitHub Actions
Copy link

@Mergifyio backport branch-3.3

@github-actions GitHub Actions
Copy link

@Mergifyio backport branch-3.2

@github-actions GitHub Actions
Copy link

@Mergifyio backport branch-3.1

@github-actions github-actions bot removed the 3.1 label Jul 10, 2024
@mergify Mergify
Copy link
Contributor

mergify bot commented Jul 10, 2024 •
edited
Loading

backport branch-3.3

✅ Backports have been created

@mergify Mergify
Copy link
Contributor

mergify bot commented Jul 10, 2024 •
edited
Loading

backport branch-3.2

✅ Backports have been created

@mergify Mergify
Copy link
Contributor

mergify bot commented Jul 10, 2024 •
edited
Loading

backport branch-3.1

✅ Backports have been created

mergify bot pushed a commit that referenced this pull request Jul 10, 2024
@meegoo @mergify
[BugFix] Fix automatic partition hang when open timeout is too long e…
365ffd0 
…xhausted automatic partition rpc thread pool (#47838)

Signed-off-by: meegoo <[email protected]>
(cherry picked from commit 3fadc53)
@mergify mergify bot mentioned this pull request Jul 10, 2024
Merged
42 tasks
mergify bot pushed a commit that referenced this pull request Jul 10, 2024
@meegoo @mergify
[BugFix] Fix automatic partition hang when open timeout is too long e…
2d0a701 
…xhausted automatic partition rpc thread pool (#47838)

Signed-off-by: meegoo <[email protected]>
(cherry picked from commit 3fadc53)

# Conflicts:
#	be/src/exec/tablet_sink_index_channel.cpp
@mergify mergify bot mentioned this pull request Jul 10, 2024
Merged
24 tasks
mergify bot pushed a commit that referenced this pull request Jul 10, 2024
@meegoo @mergify
[BugFix] Fix automatic partition hang when open timeout is too long e…
209d2ac 
…xhausted automatic partition rpc thread pool (#47838)

Signed-off-by: meegoo <[email protected]>
(cherry picked from commit 3fadc53)

# Conflicts:
#	be/src/exec/tablet_sink_index_channel.cpp
@mergify mergify bot mentioned this pull request Jul 10, 2024
Merged
42 tasks
wanpengfei-git pushed a commit that referenced this pull request Jul 10, 2024
@mergify @meegoo
[BugFix] Fix automatic partition hang when open timeout is too long e…
bd0c721 
…xhausted automatic partition rpc thread pool (backport #47838) (#48099)

Co-authored-by: meegoo <[email protected]>
meegoo added a commit that referenced this pull request Jul 10, 2024
@mergify @meegoo
[BugFix] Fix automatic partition hang when open timeout is too long e…
09b7c49 
…xhausted automatic partition rpc thread pool (backport #47838) (#48101)

Co-authored-by: meegoo <[email protected]>
meegoo added a commit that referenced this pull request Aug 5, 2024
@mergify @meegoo
[BugFix] Fix automatic partition hang when open timeout is too long e…
043181d 
…xhausted automatic partition rpc thread pool (backport #47838) (#48100)

Signed-off-by: meegoo <[email protected]>
Co-authored-by: meegoo <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wyb wyb wyb approved these changes

@kevincai kevincai kevincai approved these changes

@jaogoy jaogoy jaogoy left review comments

Assignees

@meegoo meegoo

Labels
3.1-merged 3.2-merged 3.3-merged behavior_changed documentation Improvements or additions to documentation
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@meegoo @kevincai @wyb @jaogoy