Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-32852][JUnit5 Migration] Migrate the scheduler package of flink-runtime module to JUnit5 #24732

Merged
merged 8 commits into from
May 7, 2024

Conversation

RocMarshal
Copy link
Contributor

What is the purpose of the change

[FLINK-32852][JUnit5 Migration] Migrate the scheduler package of flink-runtime module to JUnit5

Brief change log

[FLINK-32852][JUnit5 Migration] Migrate the scheduler package of flink-runtime module to JUnit5

Verifying this change

This change is already covered by existing tests, such as (please describe tests).

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): (yes / no)
  • The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
  • The serializers: (yes / no / don't know)
  • The runtime per-record code paths (performance sensitive): (yes / no / don't know)
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / no / don't know)
  • The S3 file system connector: (yes / no / don't know)

Documentation

  • Does this pull request introduce a new feature? (yes / no)
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

@flinkbot
Copy link
Collaborator

flinkbot commented Apr 27, 2024

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@RocMarshal
Copy link
Contributor Author

RocMarshal commented Apr 28, 2024

hi, @JingGe @1996fanrui Could you help take a look if you had the free time ?
Thank you very much~

Copy link
Member

@1996fanrui 1996fanrui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some classes should be changed, but they are missed, so I commented here:

  • Remove @ExtendWith(TestLoggerExtension.class)
    • EnforceMinimalIncreaseRescalingControllerTest
    • DefaultExecutionDeployerTest
    • StopWithSavepointTest
  • Don't use org.junit.Assert
    • StateValidator
    • SchedulerTestingUtils
    • MockStateWithExecutionGraphContext
  • public can be removed for test method
    • AdaptiveSchedulerTest
    • StateLocalitySlotAssignerTest
  • public can be removed for test class
  • you can search public class, I saw a lot of classes need to remove it.
  • Update isEqualTo(0) to isZero
    • AdaptiveSchedulerTest
    • AllToAllBlockingResultInfoTest
  • Update isEqualTo(1) to isOne
    • ExecutionTimeBasedSlowTaskDetectorTest
    • SpeculativeExecutionTest
    • PointwiseBlockingResultInfoTest
    • DefaultVertexParallelismAndInputInfosDeciderTest
    • AdaptiveSchedulerTest
    • AllToAllBlockingResultInfoTest
  • hasSameSizeAs
    • DefaultVertexParallelismAndInputInfosDeciderTest (line 445)

Comment on lines +120 to +139

@Override
public void beforeEach(ExtensionContext context) throws Exception {
miniClusterResource.before();
}

@Override
public void afterEach(ExtensionContext context) throws Exception {
miniClusterResource.after();
}

@Override
public void before(ExtensionContext context) throws Exception {
miniClusterResource.before();
}

@Override
public void after(ExtensionContext context) throws Exception {
miniClusterResource.after();
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I saw the mini cluster resource didn't close for each test before. I guess one reason is performance.

Start a mini cluster needs some times, if we restart it frequently, test will take a long time.

Is it necessary for this PR?

image

stateWithExecutionGraph.getGloballyTerminalStateFuture().get(),
is(JobStatus.FINISHED));
assertThatFuture(stateWithExecutionGraph.getGloballyTerminalStateFuture())
.isCompletedWithValue(JobStatus.FINISHED);
}

@Test
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All public can be removed.

finishedBackgroundTask.getTerminationFuture().join();
}

@Test
public void testFinishedBackgroundTaskDoesNotContainAResult() {
final BackgroundTask<Void> finishedBackgroundTask = BackgroundTask.finishedBackgroundTask();

assertTrue(finishedBackgroundTask.getResultFuture().isCompletedExceptionally());
assertThatFuture(finishedBackgroundTask.getResultFuture()).isCompletedExceptionally();
}

@Test
public void testNormalCompletionOfBackgroundTask() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All public can be removed.

@@ -182,21 +182,20 @@ private void triggerComputeNumOfSubpartitions(IntermediateResult result) {
private void assertNetworkMemory(
List<SlotSharingGroup> slotSharingGroups, List<MemorySize> networkMemory) {

assertEquals(slotSharingGroups.size(), networkMemory.size());
assertThat(networkMemory.size()).isEqualTo(slotSharingGroups.size());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hasSameSizeAs

Comment on lines 280 to 281
assertThat(slotSharingGroups.size()).isEqualTo(3);
assertThat(parallelisms.size()).isEqualTo(3);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hasSize

@1996fanrui 1996fanrui self-assigned this Apr 29, 2024
@RocMarshal
Copy link
Contributor Author

public can be removed for test method
AdaptiveSchedulerTest
StateLocalitySlotAssignerTest

Thank you very much for the patient-check and review.

There're some class referenced by other class. so I kept the public key-word for the related class and removed the key-word for some class.

As shown in the commit messages, I split the patches by commit.
I'd be appreciated with your next confirmation!

@1996fanrui
Copy link
Member

@flinkbot run azure

Copy link
Member

@1996fanrui 1996fanrui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @RocMarshal for the update!

LGTM

@1996fanrui 1996fanrui merged commit ca441b8 into apache:master May 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants