Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix flaking TestStoreListResourceVersion #125450

Merged
merged 1 commit into from
Jun 12, 2024
Merged

Fix flaking TestStoreListResourceVersion #125450

merged 1 commit into from
Jun 12, 2024

Conversation

mauri870
Copy link
Member

@mauri870 mauri870 commented Jun 12, 2024 •
edited
Loading

What type of PR is this?

/kind cleanup
/kind failing-test
/kind flake

What this PR does / why we need it:

$ go test -c ./staging/src/k8s.io/apiserver/pkg/registry/generic/registry
$ go run golang.org/x/tools/cmd/stress@latest -failure storage -p 1 ./registry.test -test.v -test.run ^TestStoreListResourceVersion$
5s: 9 runs so far, 0 failures
10s: 18 runs so far, 0 failures
15s: 28 runs so far, 0 failures
20s: 36 runs so far, 0 failures

After #124642 was merged it started flaking the test runs for TestStoreListResourceVersion.

KEP https://kep.k8s.io/4568 says:

... an approach where we only delegate to etcd the (list) requests that (a) are not
setting any selectors AND (b) are setting a limit.

Since this test does not set a limit in ListOptions, it ends up trying to use the watchcache instead of calling etcd. The flakiness comes from the fact that the cache is not initialized, which returns a "too many requests" error and causes the test to fail.

Fixed by waiting until the cache is ready in order to run the tests.

Which issue(s) this PR fixes:

Fixes #125406

Special notes for your reviewer:

Does this PR introduce a user-facing change?

pkg k8s.io/apiserver/pkg/storage/cacher, method (*Cacher) Wait(context.Context) error

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

[Related KEP]: https://kep.k8s.io/4568

@k8s-ci-robot k8s-ci-robot added size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. release-note-none Denotes a PR that doesn't merit a release note. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. kind/failing-test Categorizes issue or PR as related to a consistently or frequently failing test. kind/flake Categorizes issue or PR as related to a flaky test. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jun 12, 2024
@k8s-ci-robot
Copy link
Contributor

Hi @mauri870. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added needs-priority Indicates a PR lacks a `priority/foo` label and requires one. area/apiserver sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Jun 12, 2024
@mauri870 mauri870 force-pushed the hotfix/flaky-TestStoreListResourceVersion branch from 22512e7 to e59988d Compare June 12, 2024 10:12
@k8s-ci-robot k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. release-note-none Denotes a PR that doesn't merit a release note. labels Jun 12, 2024
@wojtek-t
Copy link
Member

/lgtm
/approve
/ok-to-test

/priority important-soon

Thanks for fixing this!

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Jun 12, 2024
@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 12, 2024
@k8s-ci-robot
Copy link
Contributor

LGTM label has been added.

Git tree hash: 58028eba432a0a00959ba89589cb6b9d41c8443a

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mauri870, wojtek-t

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 12, 2024
@k8s-ci-robot k8s-ci-robot merged commit 4630bef into kubernetes:master Jun 12, 2024
14 checks passed
@k8s-ci-robot k8s-ci-robot added this to the v1.31 milestone Jun 12, 2024
@mauri870 mauri870 deleted the hotfix/flaky-TestStoreListResourceVersion branch June 12, 2024 14:12
@Jefftree
Copy link
Member

/triage accepted

@k8s-ci-robot k8s-ci-robot added triage/accepted Indicates an issue or PR is ready to be actively worked on. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Jun 13, 2024
@mauri870 mauri870 mentioned this pull request Jun 14, 2024
Closed
11 tasks
@Donnerbart Donnerbart mentioned this pull request Sep 5, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@deads2k deads2k Awaiting requested review from deads2k

@mikedanese mikedanese Awaiting requested review from mikedanese

Assignees

@wojtek-t wojtek-t

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/apiserver cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. kind/failing-test Categorizes issue or PR as related to a consistently or frequently failing test. kind/flake Categorizes issue or PR as related to a flaky test. lgtm "Looks good to me", indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
None yet
Milestone
v1.31
Development

Successfully merging this pull request may close these issues.

[Flaking test] unit test TestStoreListResourceVersion
4 participants
@mauri870 @k8s-ci-robot @wojtek-t @Jefftree