Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] ValuesIntAggregatorFunctionTests classMethod failing #109932

Closed
DaveCTurner opened this issue Jun 19, 2024 · 4 comments · Fixed by #110001
Closed

[CI] ValuesIntAggregatorFunctionTests classMethod failing #109932

DaveCTurner opened this issue Jun 19, 2024 · 4 comments · Fixed by #110001
Assignees
Labels
:Analytics/ES|QL AKA ESQL needs:risk Requires assignment of a risk label (low, medium, blocker) Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) >test-failure Triaged test failures from CI

Comments

@DaveCTurner
Copy link
Contributor

Build scan:
https://gradle-enterprise.elastic.co/s/be7jmqtelosou/tests/:x-pack:plugin:esql:compute:test/org.elasticsearch.compute.aggregation.ValuesIntAggregatorFunctionTests

Reproduction line:

null

Applicable branches:
main

Reproduces locally?:
Didn't try

Failure history:
Failure dashboard for org.elasticsearch.compute.aggregation.ValuesIntAggregatorFunctionTests#classMethod

Failure excerpt:

java.lang.Exception: Suite timeout exceeded (>= 1200000 msec).

  at __randomizedtesting.SeedInfo.seed([EDC079DF42C286BD]:0)

@DaveCTurner DaveCTurner added :Analytics/ES|QL AKA ESQL >test-failure Triaged test failures from CI labels Jun 19, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

@elasticsearchmachine elasticsearchmachine added Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) needs:risk Requires assignment of a risk label (low, medium, blocker) labels Jun 19, 2024
DaveCTurner added a commit that referenced this issue Jun 19, 2024
@nik9000
Copy link
Member

nik9000 commented Jun 19, 2024

null

Amazing

@nik9000
Copy link
Member

nik9000 commented Jun 19, 2024

A bunch of tests ran mega slow and timed out. That's what we get I guess.

It doesn't reproduce locally. I'll poke more tomorrow when I'm back.

@nik9000 nik9000 self-assigned this Jun 19, 2024
nik9000 added a commit to nik9000/elasticsearch that referenced this issue Jun 20, 2024
It's really slow. Like, can take a minute per test. Most of that time
was in `containsInAnyOrder`. This replaces that with a check for a `Set`
containing stuff.

This came up because we hit a timeout on one of the values tests. I
don't think these tests caused the timeout, but we may as well speed
them up.

Closes elastic#109932
@nik9000
Copy link
Member

nik9000 commented Jun 20, 2024

Most tests are running fast but ValuesIntAgg is slow here:

                     testMultivaluedWithNulls  93.727
                         testManyInitialFinal  39.214
                 testInitialIntermediateFinal 165.748
              testManyInitialManyPartialFinal 136.727
                    testNullIntermediateFinal   0.024
        testManyInitialManyPartialFinalRunner 353.345
                         testSimpleLargeInput   0.523
                   testEmptyInputInitialFinal   0.145
                         testSimpleWithCranky   0.068
       testEmptyInputInitialIntermediateFinal   0.058
                             testInitialFinal 378.527
                        testSimpleDescription   0.089
                               testEmptyInput   0.169
                             testIgnoresNulls   0.941
testManyInitialManyPartialFinalRunnerThrowing   0.285
                              testMultivalued  30.53

This doesn't reproduce. At all. It takes 20 seconds locally.

Here's the logging for some of the slow ones:

[2024-06-19T15:30:31,940][INFO ][o.e.c.a.ValuesIntAggregatorFunctionTests] [testInitialFinal] before test
[2024-06-19T15:36:50,455][INFO ][o.e.c.a.ValuesIntAggregatorFunctionTests] [testInitialFinal] after test
[2024-06-19T15:24:37,801][INFO ][o.e.c.a.ValuesIntAggregatorFunctionTests] [testManyInitialManyPartialFinalRunner] before test
[2024-06-19T15:30:31,127][INFO ][o.e.c.a.ValuesIntAggregatorFunctionTests] [testManyInitialManyPartialFinalRunner] after test

Here's the stacktrace around the timeout:

  2> 	at app//org.hamcrest.core.IsEqual.areEqual(IsEqual.java:40)
  2> 	at app//org.hamcrest.core.IsEqual.matches(IsEqual.java:23)
  2> 	at app//org.hamcrest.collection.IsIterableContainingInAnyOrder$Matching.isMatched(IsIterableContainingInAnyOrder.java:69)
  2> 	at app//org.hamcrest.collection.IsIterableContainingInAnyOrder$Matching.matches(IsIterableContainingInAnyOrder.java:54)
  2> 	at app//org.hamcrest.collection.IsIterableContainingInAnyOrder.matchesSafely(IsIterableContainingInAnyOrder.java:25)
  2> 	at app//org.hamcrest.collection.IsIterableContainingInAnyOrder.matchesSafely(IsIterableContainingInAnyOrder.java:14)
  2> 	at app//org.hamcrest.TypeSafeDiagnosingMatcher.matches(TypeSafeDiagnosingMatcher.java:57)
  2> 	at app//org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:10)
  2> 	at app//org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:6)
  2> 	at app//org.elasticsearch.test.ESTestCase.assertThat(ESTestCase.java:2303)
  2> 	at app//org.elasticsearch.compute.aggregation.ValuesIntAggregatorFunctionTests.assertSimpleOutput(ValuesIntAggregatorFunctionTests.java:41)
  2> 	at app//org.elasticsearch.compute.aggregation.Aggrega  2> torFunctionTestCase.assertSimpleOutput(AggregatorFunctionTestCase.java:82)
  2> 	at app//org.elasticsearch.compute.aggregation.AggregatorFunctionTestCase.testMultivalued(AggregatorFunctionTestCase.java:115)

I've opened a PR (#110001) to speed up the tests. I don't think the timeout is actually the fault of the test, but it's something.

javanna pushed a commit to javanna/elasticsearch that referenced this issue Jun 21, 2024
It's really slow. Like, can take a minute per test. Most of that time
was in `containsInAnyOrder`. This replaces that with a check for a `Set`
containing stuff.

This came up because we hit a timeout on one of the values tests. I
don't think these tests caused the timeout, but we may as well speed
them up.

Closes elastic#109932
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/ES|QL AKA ESQL needs:risk Requires assignment of a risk label (low, medium, blocker) Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) >test-failure Triaged test failures from CI
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants