YARN-8234. Improve RM system metrics publisher's performance by pushi… #3793

ashutoshcipher · 2021-12-13T13:11:25Z

Description of PR
Jira: https://issues.apache.org/jira/browse/YARN-8234
Improve RM system metrics publisher's performance by pushing events to timeline server in batch.

The patch is tested by applying it in-house clusters.

…ng events to timeline server in batch.

ashutoshcipher · 2021-12-14T02:35:21Z

@aajisaka , please review. Thanks

aajisaka

Thank you @ashutoshcipher for providing the patch. Some comments and questions:

Would you provide a new test case for this feature?
Would you fix the checkstyle warnings?
Can we use SLF4J parameterized logging format as possible?

In addition, I have a question. Why YarnConfiguration.RM_TIMELINE_SERVER_V1_PUBLISHER_INTERVAL is set to 1 in some tests?

...n/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/TimelineServiceV1Publisher.java

ashutoshcipher · 2021-12-17T10:51:03Z

Thank you @ashutoshcipher for providing the patch. Some comments and questions:

Would you provide a new test case for this feature?

Would you fix the checkstyle warnings?

Can we use SLF4J parameterized logging format as possible?

In addition, I have a question. Why YarnConfiguration.RM_TIMELINE_SERVER_V1_PUBLISHER_INTERVAL is set to 1 in some tests?

Would you provide a new test case for this feature?

Writing test case for this is tricky, may we can do that later. Also current tests cases are making sure the functionality is overall intact.

Would you fix the checkstyle warnings?

I will do that.

Can we use SLF4J parameterized logging format as possible?

I will do that.

In addition, I have a question. Why YarnConfiguration.RM_TIMELINE_SERVER_V1_PUBLISHER_INTERVAL is set to 1 in some tests?

This param was introduced as part of this change, so updated in test cases with value 1 as internal to make sure overall functionality is intact for tests to check.

aajisaka · 2021-12-17T15:59:07Z

Also current tests cases are making sure the functionality is overall intact.

It is not true because the feature is not enabled by default. Would you create a new test case with the feature enabled?

This param was introduced as part of this change, so updated in test cases with value 1 as internal to make sure overall functionality is intact for tests to check.

By default, setting this parameter does not take effect.

ashutoshcipher · 2021-12-19T15:04:57Z

Also current tests cases are making sure the functionality is overall intact.

It is not true because the feature is not enabled by default. Would you create a new test case with the feature enabled?

I am creating a new class TestSystemMetricsPublisherInBatch extending the TestSystemMetricsPublisher class, enabling the feature and adding these params RM_TIMELINE_SERVER_V1_PUBLISHER_BATCH_ENABLED and RM_TIMELINE_SERVER_V1_PUBLISHER_INTERVAL while setup

This param was introduced as part of this change, so updated in test cases with value 1 as internal to make sure overall functionality is intact for tests to check.

By default, setting this parameter does not take effect.

Yes, agreed.

hadoop-yetus · 2021-12-19T22:37:54Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 46s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 4 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 49s		Maven dependency ordering for branch
+1 💚	mvninstall	22m 35s		trunk passed
+1 💚	compile	10m 26s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	8m 51s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	1m 39s		trunk passed
+1 💚	mvnsite	3m 31s		trunk passed
+1 💚	javadoc	2m 55s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	2m 36s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	6m 12s		trunk passed
+1 💚	shadedclient	21m 40s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 27s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 17s		the patch passed
+1 💚	compile	9m 44s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	9m 44s		the patch passed
+1 💚	compile	8m 43s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	8m 43s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	1m 43s	/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt	hadoop-yarn-project/hadoop-yarn: The patch generated 4 new + 187 unchanged - 1 fixed = 191 total (was 188)
+1 💚	mvnsite	3m 12s		the patch passed
+1 💚	xml	0m 2s		The patch has no ill-formed XML file.
+1 💚	javadoc	2m 34s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	2m 27s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	6m 26s		the patch passed
+1 💚	shadedclient	21m 49s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 5s		hadoop-yarn-api in the patch passed.
+1 💚	unit	5m 0s		hadoop-yarn-common in the patch passed.
+1 💚	unit	97m 7s		hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚	asflicense	0m 55s		The patch does not generate ASF License warnings.
		258m 29s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3793/4/artifact/out/Dockerfile
GITHUB PR	#3793
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell xml
uname	Linux c5c8c0e21153 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 7b57f71b37cfc915c6b393a800198dabc4712b61
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3793/4/testReport/
Max. process+thread count	949 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3793/4/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

...n/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/TimelineServiceV1Publisher.java

...org/apache/hadoop/yarn/server/resourcemanager/metrics/TestSystemMetricsPublisherInBatch.java

aajisaka · 2021-12-20T07:21:33Z

Would you address the above comments and fix the checkstyle warnings? I'm +1 if all of those are addressed. Thanks.

...t/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/TestSystemMetricsPublisher.java

...org/apache/hadoop/yarn/server/resourcemanager/metrics/TestSystemMetricsPublisherInBatch.java

hadoop-yetus · 2021-12-20T14:02:13Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 42s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 4 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 43s		Maven dependency ordering for branch
+1 💚	mvninstall	23m 11s		trunk passed
+1 💚	compile	9m 42s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	8m 40s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	1m 45s		trunk passed
+1 💚	mvnsite	3m 13s		trunk passed
+1 💚	javadoc	2m 45s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	2m 29s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	5m 57s		trunk passed
+1 💚	shadedclient	21m 45s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 28s		Maven dependency ordering for patch
-1 ❌	mvninstall	0m 49s	/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt	hadoop-yarn-server-resourcemanager in the patch failed.
-1 ❌	compile	3m 43s	/patch-compile-hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt	hadoop-yarn in the patch failed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.
-1 ❌	javac	3m 43s	/patch-compile-hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt	hadoop-yarn in the patch failed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.
-1 ❌	compile	3m 9s	/patch-compile-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt	hadoop-yarn in the patch failed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.
-1 ❌	javac	3m 9s	/patch-compile-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt	hadoop-yarn in the patch failed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	1m 20s	/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt	hadoop-yarn-project/hadoop-yarn: The patch generated 2 new + 187 unchanged - 1 fixed = 189 total (was 188)
-1 ❌	mvnsite	0m 52s	/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt	hadoop-yarn-server-resourcemanager in the patch failed.
+1 💚	xml	0m 1s		The patch has no ill-formed XML file.
+1 💚	javadoc	1m 56s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 45s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
-1 ❌	spotbugs	0m 52s	/patch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt	hadoop-yarn-server-resourcemanager in the patch failed.
-1 ❌	shadedclient	9m 41s		patch has errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	0m 51s		hadoop-yarn-api in the patch passed.
+1 💚	unit	4m 36s		hadoop-yarn-common in the patch passed.
-1 ❌	unit	0m 52s	/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt	hadoop-yarn-server-resourcemanager in the patch failed.
+1 💚	asflicense	0m 31s		The patch does not generate ASF License warnings.
		130m 59s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3793/10/artifact/out/Dockerfile
GITHUB PR	#3793
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell xml
uname	Linux f6f24488f6a3 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 51a10db4b402f5bc07baf1937b79c06b33d1df58
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3793/10/testReport/
Max. process+thread count	601 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3793/10/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-12-20T16:34:22Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 46s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 4 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	13m 16s		Maven dependency ordering for branch
+1 💚	mvninstall	21m 46s		trunk passed
+1 💚	compile	9m 41s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	8m 25s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	1m 45s		trunk passed
+1 💚	mvnsite	3m 16s		trunk passed
+1 💚	javadoc	2m 57s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	2m 44s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	5m 57s		trunk passed
+1 💚	shadedclient	21m 58s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 28s		Maven dependency ordering for patch
-1 ❌	mvninstall	0m 48s	/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt	hadoop-yarn-server-resourcemanager in the patch failed.
-1 ❌	compile	3m 43s	/patch-compile-hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt	hadoop-yarn in the patch failed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.
-1 ❌	javac	3m 43s	/patch-compile-hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt	hadoop-yarn in the patch failed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.
-1 ❌	compile	3m 8s	/patch-compile-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt	hadoop-yarn in the patch failed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.
-1 ❌	javac	3m 8s	/patch-compile-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt	hadoop-yarn in the patch failed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	1m 23s		hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 187 unchanged - 1 fixed = 187 total (was 188)
-1 ❌	mvnsite	0m 53s	/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt	hadoop-yarn-server-resourcemanager in the patch failed.
+1 💚	xml	0m 2s		The patch has no ill-formed XML file.
+1 💚	javadoc	1m 54s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 48s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
-1 ❌	spotbugs	0m 50s	/patch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt	hadoop-yarn-server-resourcemanager in the patch failed.
-1 ❌	shadedclient	9m 34s		patch has errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	0m 52s		hadoop-yarn-api in the patch passed.
+1 💚	unit	4m 43s		hadoop-yarn-common in the patch passed.
-1 ❌	unit	0m 52s	/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt	hadoop-yarn-server-resourcemanager in the patch failed.
+1 💚	asflicense	0m 33s		The patch does not generate ASF License warnings.
		130m 43s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3793/11/artifact/out/Dockerfile
GITHUB PR	#3793
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell xml
uname	Linux 6ce192781838 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 346d8df111c73c448359721b1ac1744bdb97798c
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3793/11/testReport/
Max. process+thread count	542 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3793/11/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

aajisaka · 2021-12-21T05:48:02Z

Maybe smart-apply-patch is somewhat broken. Would you rebase the commits related to the test and force-push it?

…ure (apache#3774) Reviewed-by: Masatake Iwasaki <[email protected]>

…h class. Contributed by Tibor Kovacs

…ssage. (apache#3599) Change-Id: I070fc4784679b3be73aa3a11201bbae23c20ad4e

… queues. Contributed by Tamas Domok

Signed-off-by: Akira Ajisaka <[email protected]>

…own (apache#3768) Signed-off-by: Akira Ajisaka <[email protected]>

…tive asserts. Contributed by Benjamin Teke

Reviewed-by: Viraj Jasani <[email protected]> Signed-off-by: Takanobu Asanuma <[email protected]>

This reverts commit a4557f9.

…ation API (apache#3799). Contributed by Szilard Nemeth

…ed by Tamas Domok

…hat belongs to async scheduling to a new class (apache#3800). Contributed by Szilard Nemeth

…d by Szilard Nemeth

…Report (apache#3714). Contributed by liubingxing. Signed-off-by: He Xiaoqiao <[email protected]>

…ect (apache#3775) Signed-off-by: Akira Ajisaka <[email protected]>

…ge for future (apache#3789) Signed-off-by: Akira Ajisaka <[email protected]>

…e when ATSv2 is enabled (apache#3802)

…#3815)

…rking. (apache#3806)

hadoop-yetus · 2021-12-22T12:13:46Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 0s		Docker mode activated.
-1 ❌	patch	0m 20s		#3793 does not apply to trunk. Rebase required? Wrong Branch? See https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute for help.

Subsystem	Report/Notes
GITHUB PR	#3793
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3793/12/console
versions	git=2.17.1
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

ashutoshcipher · 2021-12-22T12:15:07Z

Maybe smart-apply-patch is somewhat broken. Would you rebase the commits related to the test and force-push it?

Sure, I will do that

hadoop-yetus · 2021-12-22T16:33:31Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 41s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 52s		Maven dependency ordering for branch
+1 💚	mvninstall	21m 44s		trunk passed
+1 💚	compile	9m 46s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	8m 26s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	1m 43s		trunk passed
+1 💚	mvnsite	3m 16s		trunk passed
+1 💚	javadoc	2m 56s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	2m 47s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	6m 8s		trunk passed
+1 💚	shadedclient	22m 24s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	22m 46s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 28s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 13s		the patch passed
+1 💚	compile	9m 27s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	9m 27s		the patch passed
+1 💚	compile	8m 47s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	8m 47s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	1m 33s		hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 167 unchanged - 1 fixed = 167 total (was 168)
+1 💚	mvnsite	2m 51s		the patch passed
+1 💚	xml	0m 2s		The patch has no ill-formed XML file.
+1 💚	javadoc	2m 30s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	2m 27s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	6m 19s		the patch passed
+1 💚	shadedclient	22m 16s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 6s		hadoop-yarn-api in the patch passed.
+1 💚	unit	4m 50s		hadoop-yarn-common in the patch passed.
+1 💚	unit	98m 45s		hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚	asflicense	0m 54s		The patch does not generate ASF License warnings.
		258m 19s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3793/13/artifact/out/Dockerfile
GITHUB PR	#3793
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell xml
uname	Linux df9ac0ac06f6 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `c7ba4e4`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3793/13/testReport/
Max. process+thread count	936 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3793/13/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

aajisaka · 2021-12-23T08:17:17Z

Merged. Thank you @ashutoshcipher for your contribution!

…ng events to timeline server in batch (#3793) Signed-off-by: Akira Ajisaka <[email protected]> (cherry picked from commit 00e2405)

…ng events to timeline server in batch (#3793) Signed-off-by: Akira Ajisaka <[email protected]> (cherry picked from commit 00e2405) Conflicts: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/TestSystemMetricsPublisher.java

…ng events to timeline server in batch (#3793) Signed-off-by: Akira Ajisaka <[email protected]> (cherry picked from commit 00e2405)

…ng events to timeline server in batch (apache#3793) Signed-off-by: Akira Ajisaka <[email protected]>

YARN-8234. Improve RM system metrics publisher's performance by pushi…

27643d5

…ng events to timeline server in batch.

This comment has been minimized.

Sign in to view

aajisaka self-requested a review December 14, 2021 07:19

aajisaka requested changes Dec 14, 2021

View reviewed changes

This comment has been minimized.

Sign in to view