Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Transform] Always pick the user maxPageSize value #109876

Merged
merged 5 commits into from
Jun 21, 2024

Conversation

prwhelan
Copy link
Member

maxPageSize's update functionality has been reprioritized. If a user calls the update API to change the max page size, that update will lock out other threads from changing the max page size. Once the lock is released, the other threads will check again if they can reset the max page size and otherwise keep the value that the user just updated with.

Fix #109844

maxPageSize's update functionality has been reprioritized.  If a user
calls the update API to change the max page size, that update will lock
out other threads from changing the max page size.  Once the lock is
released, the other threads will check again if they can reset the max
page size and otherwise keep the value that the user just updated with.

Fix elastic#109844
@prwhelan prwhelan added >bug :ml/Transform Transform Team:ML Meta label for the ML team labels Jun 18, 2024
@elasticsearchmachine
Copy link
Collaborator

Hi @prwhelan, I've created a changelog YAML for you.

@prwhelan
Copy link
Member Author

@elasticmachine update branch

@prwhelan prwhelan marked this pull request as ready for review June 18, 2024 19:14
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/ml-core (Team:ML)

Copy link
Contributor

@jan-elastic jan-elastic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@jonathan-buttner jonathan-buttner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it worth writing a test for the changes? Looks like this was to fix a failing test so maybe that's good enough?

@prwhelan
Copy link
Member Author

prwhelan commented Jun 21, 2024

Is it worth writing a test for the changes? Looks like this was to fix a failing test so maybe that's good enough?

Well I wrote one, I'm not sure if it's a good one, but I'll update this PR and we can always revert it if we don't like it.

I tend to prefer when expectations are represented in tests, so I'm inclined to include it, but it requires knowing how the indexer is implemented so it may be obnoxious to maintain so I'm okay with omitting it (which was my initial thought)

@prwhelan
Copy link
Member Author

@elasticmachine update branch

@prwhelan prwhelan merged commit c0f2081 into elastic:main Jun 21, 2024
15 checks passed
@davidkyle
Copy link
Member

The related test is failing on 8.14, should this fix be backported?

https://gradle-enterprise.elastic.co/s/iwv7otxdqb64w/tests/:x-pack:plugin:transform:test/org.elasticsearch.xpack.transform.transforms.TransformIndexerTests/testMaxPageSearchSizeIsResetToConfiguredValue

prwhelan added a commit to prwhelan/elasticsearch that referenced this pull request Jul 29, 2024
maxPageSize's update functionality has been reprioritized.  If a user
calls the update API to change the max page size, that update will lock
out other threads from changing the max page size.  Once the lock is
released, the other threads will check again if they can reset the max
page size and otherwise keep the value that the user just updated with.

Relate elastic#109844
Fix elastic#109876
prwhelan added a commit that referenced this pull request Jul 29, 2024
maxPageSize's update functionality has been reprioritized.  If a user
calls the update API to change the max page size, that update will lock
out other threads from changing the max page size.  Once the lock is
released, the other threads will check again if they can reset the max
page size and otherwise keep the value that the user just updated with.

Relate #109844
Fix #109876
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>bug :ml/Transform Transform Team:ML Meta label for the ML team v8.15.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[CI] TransformIndexerTests testMaxPageSearchSizeIsResetToConfiguredValue failing
6 participants