Speed up windows CI/CD #1982

wcs1only · 2020-08-31T23:46:04Z

Description

This PR contains several improvements to E2E tests for both linux and windows tests.

Rather than do a multistaged windows build from servercore and nanoserver, we'll use a prebuilt nanoserver image that already has the dependancies we need. This saves us 5-10 minutes.
Don't wait for container termination to start the next test. Container termination, in some cases, takes longer than the actual test. This saves us ~10 minutes.
Build the e2e test apps on the host machine instead of in an intermediate golang container. One of the big drags of the e2e docker container build was pulling all the go dependancies each time for each app. Building on the host machine allows dependancies to be pulled only once. This saves us nearly 10 minutes on the windows build.
Run 2 test apps at a time. Since each test app is independent from all the others, we can run more that one without issue. This saves us ~10-20 minutes.

These improvements gets the tests down to ~20 minutes for linux and ~30 minutes for windows. Since this is even better than our current test runtime (~30-40 minutes) we should be able to run both in parallel each time we do /ok-to-test

Issue reference

We strive to have all PR being opened based on an issue, where the problem or feature have been discussed prior to implementation.

Please reference the issue this PR will close: #1944

Checklist

Please make sure you've completed the relevant tasks for this PR, out of the following list:

Code compiles correctly
Created/updated tests
Unit tests passing
End-to-end tests passing
Extended the documentation / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number]
Specification has been updated / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number]
Provided sample for the feature / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number]

wcs1only · 2020-08-31T23:46:15Z

/ok-to-test

wcs1only · 2020-08-31T23:46:28Z

/ok-to-wintest

dapr-bot · 2020-08-31T23:47:04Z

Found the available test cluster - dapr-aks-e2e-05. Please wait until test is done.

dapr-bot · 2020-08-31T23:48:17Z

Found the available test cluster - dapr-aks-e2e-06. Please wait until test is done.

dapr-bot · 2020-08-31T23:53:05Z

End-to-end tests failed.

wcs1only · 2020-09-01T00:11:41Z

/ok-to-wintest

dapr-bot · 2020-09-01T00:14:01Z

Found the available test cluster - dapr-aks-e2e-06. Please wait until test is done.

dapr-bot · 2020-09-01T00:14:31Z

End-to-end tests failed.

dapr-bot · 2020-09-01T00:38:29Z

End-to-end tests failed.

wcs1only · 2020-09-01T02:05:07Z

/ok-to-test

dapr-bot · 2020-09-01T02:05:59Z

Found the available test cluster - dapr-aks-e2e-06. Please wait until test is done.

dapr-bot · 2020-09-01T02:26:44Z

Congrats! All end-to-end tests have passed. Thanks for your contribution!

wcs1only · 2020-09-01T02:27:05Z

/ok-to-wintest

dapr-bot · 2020-09-01T02:28:37Z

Found the available test cluster - dapr-aks-e2e-05. Please wait until test is done.

dapr-bot · 2020-09-01T02:33:38Z

End-to-end tests failed.

wcs1only · 2020-09-01T14:48:04Z

/ok-to-wintest

dapr-bot · 2020-09-01T14:50:11Z

Found the available test cluster - dapr-aks-e2e-05. Please wait until test is done.

dapr-bot · 2020-09-01T14:55:39Z

End-to-end tests failed.

wcs1only · 2020-09-01T16:29:18Z

/ok-to-wintest

dapr-bot · 2020-09-01T16:31:04Z

Found the available test cluster - dapr-aks-e2e-05. Please wait until test is done.

dapr-bot · 2020-09-01T16:58:18Z

Congrats! All end-to-end tests have passed. Thanks for your contribution!

…dapr#1944)

tests/dapr_tests.mk

.github/workflows/dapr-bot.yml

wcs1only · 2020-09-03T16:18:23Z

/ok-to-perf

dapr-bot · 2020-09-03T16:19:18Z

Found the available test cluster - dapr-aks-e2e-05. Please wait until test is done.

dapr-bot · 2020-09-03T16:28:47Z

All Dapr perf tests completed.

youngbupark · 2020-09-03T18:08:25Z

tests/dapr_tests.mk


 # start all perf tests
 test-perf-all: check-e2e-env
- DAPR_TEST_NAMESPACE=$(DAPR_TEST_NAMESPACE) DAPR_TEST_TAG=$(DAPR_TEST_TAG) DAPR_TEST_REGISTRY=$(DAPR_TEST_REGISTRY) DAPR_TEST_MINIKUBE_IP=$(MINIKUBE_NODE_IP) go test -p 1 -count=1 -v -tags=perf ./tests/perf/...
+ DAPR_TEST_NAMESPACE=$(DAPR_TEST_NAMESPACE) DAPR_TEST_TAG=$(DAPR_TEST_TAG) DAPR_TEST_REGISTRY=$(DAPR_TEST_REGISTRY) DAPR_TEST_MINIKUBE_IP=$(MINIKUBE_NODE_IP) go test -p 2 -count=1 -v -tags=perf ./tests/perf/...


We need to keep it as 1 for perf test :) otherwise, two perf test can run in parallel, which results in the wrong perf number.

cc/ @yaron2

Good point. Will fix.

wcs1only · 2020-09-03T19:00:11Z

/ok-to-test

dapr-bot · 2020-09-03T19:00:58Z

Found the available test cluster - dapr-aks-e2e-05 for linux. Please wait until test is done.

dapr-bot · 2020-09-03T19:02:18Z

Found the available test cluster - dapr-aks-e2e-06 for windows. Please wait until test is done.

dapr-bot · 2020-09-03T19:06:05Z

End-to-end tests failed on linux.

dapr-bot · 2020-09-03T19:06:26Z

End-to-end tests cancelled on windows.

wcs1only · 2020-09-03T19:07:41Z

/ok-to-test

dapr-bot · 2020-09-03T19:08:22Z

Found the available test cluster - dapr-aks-e2e-05 for linux. Please wait until test is done.

dapr-bot · 2020-09-03T19:09:45Z

Found the available test cluster - dapr-aks-e2e-06 for windows. Please wait until test is done.

dapr-bot · 2020-09-03T19:28:44Z

Congrats! All end-to-end tests have passed on linux. Thanks for your contribution!

dapr-bot · 2020-09-03T19:37:21Z

End-to-end tests failed on windows.

wcs1only force-pushed the speed-up-windows-ci branch from 4035ef6 to a847a3b Compare September 1, 2020 00:11

wcs1only force-pushed the speed-up-windows-ci branch from a847a3b to a1f48d1 Compare September 1, 2020 02:04

wcs1only force-pushed the speed-up-windows-ci branch from a1f48d1 to cd08e67 Compare September 1, 2020 16:28

wcs1only requested a review from youngbupark September 1, 2020 18:24

wcs1only marked this pull request as ready for review September 1, 2020 18:24

Use prebuilt nanoserver images that have the appropriate dependancies (…

7305c0b

…dapr#1944)

wcs1only force-pushed the speed-up-windows-ci branch from cd08e67 to 1af765b Compare September 1, 2020 19:59

wcs1only requested review from yaron2 and vinayada1 September 1, 2020 22:05

wcs1only requested a review from artursouza September 1, 2020 22:05

artursouza reviewed Sep 2, 2020

View reviewed changes

tests/dapr_tests.mk Show resolved Hide resolved

youngbupark reviewed Sep 3, 2020

View reviewed changes

.github/workflows/dapr-bot.yml Show resolved Hide resolved

wcs1only added 2 commits September 3, 2020 08:51

Do not wait for service termination before starting the next test

e837e54

Run windows and linux tests in parallel

f3bd9c0

wcs1only force-pushed the speed-up-windows-ci branch from 1af765b to f3bd9c0 Compare September 3, 2020 15:51

Merge branch 'master' into speed-up-windows-ci

c71114d

wcs1only requested a review from youngbupark September 3, 2020 16:52

youngbupark suggested changes Sep 3, 2020

View reviewed changes

wcs1only added 2 commits September 3, 2020 11:17

Drop perf test parallelism to 1

744b447

Merge branch 'master' into speed-up-windows-ci

2a5793e

youngbupark approved these changes Sep 3, 2020

View reviewed changes

Merge branch 'master' into speed-up-windows-ci

9346ba4

wcs1only merged commit 15a3113 into dapr:master Sep 3, 2020

Speed up windows CI/CD #1982

Speed up windows CI/CD #1982

Conversation

wcs1only commented Aug 31, 2020 • edited Loading

Description

Issue reference

Checklist

wcs1only commented Aug 31, 2020

wcs1only commented Aug 31, 2020

dapr-bot commented Aug 31, 2020

dapr-bot commented Aug 31, 2020

dapr-bot commented Aug 31, 2020

wcs1only commented Sep 1, 2020

dapr-bot commented Sep 1, 2020

dapr-bot commented Sep 1, 2020

dapr-bot commented Sep 1, 2020

wcs1only commented Sep 1, 2020

dapr-bot commented Sep 1, 2020

dapr-bot commented Sep 1, 2020

wcs1only commented Sep 1, 2020

dapr-bot commented Sep 1, 2020

dapr-bot commented Sep 1, 2020

wcs1only commented Sep 1, 2020

dapr-bot commented Sep 1, 2020

dapr-bot commented Sep 1, 2020

wcs1only commented Sep 1, 2020

dapr-bot commented Sep 1, 2020

dapr-bot commented Sep 1, 2020

wcs1only commented Sep 3, 2020

dapr-bot commented Sep 3, 2020

dapr-bot commented Sep 3, 2020

youngbupark Sep 3, 2020

Choose a reason for hiding this comment

wcs1only Sep 3, 2020

Choose a reason for hiding this comment

wcs1only commented Sep 3, 2020

dapr-bot commented Sep 3, 2020

dapr-bot commented Sep 3, 2020

dapr-bot commented Sep 3, 2020

dapr-bot commented Sep 3, 2020

wcs1only commented Sep 3, 2020

dapr-bot commented Sep 3, 2020

dapr-bot commented Sep 3, 2020

dapr-bot commented Sep 3, 2020

dapr-bot commented Sep 3, 2020

wcs1only commented Aug 31, 2020 •

edited

Loading