Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[chore]run govulncheck with multiple actions #21524

Merged
merged 3 commits into from
May 6, 2023

Conversation

atoulme
Copy link
Contributor

@atoulme atoulme commented May 6, 2023

Try to run govulncheck with multiple runners in parallel to avoid hitting memory limits.

  • Split the runner across the usual boundaries of receivers, exporters, connectors, etc groups.
  • Identify that the cmd/* go modules and root create a spike in memory that triggers OOM, those are now skipped. We create a pkg group to track all pkg/* go.mod.

@atoulme atoulme requested a review from a team as a code owner May 6, 2023 06:18
@atoulme atoulme requested a review from bogdandrutu May 6, 2023 06:18
Copy link
Member

@dmitryax dmitryax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like we have a bunch issues uncovered? They can be fixed separately I think

@atoulme
Copy link
Contributor Author

atoulme commented May 6, 2023

Yes, but it looks like they are tied to our version of golang, trying to move to the latest version of go 1.19 to see if we still see those issues.

@atoulme atoulme force-pushed the parallelize_govulncheck branch 2 times, most recently from 96d4a12 to 624b6f4 Compare May 6, 2023 15:13
@atoulme
Copy link
Contributor Author

atoulme commented May 6, 2023

I still see kills on cmd/otelcontribcol and cmd/configschema:

Running target 'govulncheck' in module 'cmd/configschema' as part of group 'all'
make -C cmd/configschema govulncheck
Running target 'govulncheck' in module 'cmd/mdatagen' as part of group 'all'
make -C cmd/mdatagen govulncheck
make[1]: Entering directory '/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/cmd/configschema'
make[1]: Entering directory '/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/cmd/mdatagen'
/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/.tools/govulncheck ./...
govulncheck is an experimental tool. Share feedback at https://go.dev/s/govulncheck-feedback.

Using go1.1[9](https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/4902282363/jobs/8754032110#step:6:10).9 and [email protected] with
vulnerability data from https://vuln.go.dev (last modified 2023-05-05 21:[10](https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/4902282363/jobs/8754032110#step:6:11):24 +0000 UTC).
/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/.tools/govulncheck ./...
govulncheck is an experimental tool. Share feedback at https://go.dev/s/govulncheck-feedback.

Using go1.19.9 and [email protected] with
vulnerability data from https://vuln.go.dev (last modified 2023-05-05 21:10:24 +0000 UTC).

Scanning your code and 3[12](https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/4902282363/jobs/8754032110#step:6:13) packages across 30 dependent modules for known vulnerabilities...
No vulnerabilities found.
make[1]: Leaving directory '/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/cmd/mdatagen'
Running target 'govulncheck' in module 'cmd/otelcontribcol' as part of group 'all'
make -C cmd/otelcontribcol govulncheck
make[1]: Entering directory '/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/cmd/otelcontribcol'
/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/.tools/govulncheck ./...
govulncheck is an experimental tool. Share feedback at https://go.dev/s/govulncheck-feedback.

Using go1.19.9 and [email protected] with
vulnerability data from https://vuln.go.dev (last modified 2023-05-05 21:10:24 +0000 UTC).
make[1]: *** [../../Makefile.Common:[15](https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/4902282363/jobs/8754032110#step:6:16)6: govulncheck] Killed
make: *** [Makefile:[16](https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/4902282363/jobs/8754032110#step:6:17)7: cmd/configschema] Terminated
make[1]: *** [../../Makefile.Common:156: govulncheck] Terminated

For now, we can ignore those folders.

@dmitryax
Copy link
Member

dmitryax commented May 6, 2023

I still see kills on cmd/otelcontribcol and cmd/configschema

Because they depend on the whole repo lol. We should probably exclude them

Makefile Outdated Show resolved Hide resolved
Co-authored-by: Dmitrii Anoshin <[email protected]>
@dmitryax dmitryax merged commit 58a8964 into open-telemetry:main May 6, 2023
77 checks passed
@github-actions github-actions bot added this to the next release milestone May 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants