Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better toolchain downloader #9941

Conversation

dmcilvaney
Copy link
Contributor

@dmcilvaney dmcilvaney commented Jul 26, 2024

Merge Checklist

All boxes should be checked before merging the PR (just tick any boxes which don't apply to this PR)

  • The toolchain has been rebuilt successfully (or no changes were made to it)
  • The toolchain/worker package manifests are up-to-date
  • Any updated packages successfully build (or no packages were changed)
  • Packages depending on static components modified in this PR (Golang, *-static subpackages, etc.) have had their Release tag incremented.
  • Package tests (%check section) have been verified with RUN_CHECK=y for existing SPEC files, or added to new SPEC files
  • All package sources are available
  • cgmanifest files are up-to-date and sorted (./cgmanifest.json, ./toolkit/scripts/toolchain/cgmanifest.json, .github/workflows/cgmanifest.json)
  • LICENSE-MAP files are up-to-date (./SPECS/LICENSES-AND-NOTICES/data/licenses.json, ./SPECS/LICENSES-AND-NOTICES/LICENSES-MAP.md, ./SPECS/LICENSES-AND-NOTICES/LICENSE-EXCEPTIONS.PHOTON)
  • All source files have up-to-date hashes in the *.signatures.json files
  • sudo make go-tidy-all and sudo make go-test-coverage pass
  • Documentation has been updated to match any changes to the build system
  • If you are adding/removing a .spec file that has multiple-versions supported, please add @microsoft/cbl-mariner-multi-package-reviewers team as reviewer (Eg. golang has 2 versions 1.18, 1.21+)
  • Ready to merge

Summary

In preparation for a new explicit validation mechanism break out the toolchain downloader logic into a real script.

The new script will handle the logic that was previously baked into the makefile, while also addressing the issue of log overwriting.

Change Log
  • Move toolchain download logic into download_toolchain_rpms.sh
Does this affect the toolchain?

YES

Associated issues
  • #xxxx
Links to CVEs
Test Methodology
  • Pipeline build id: xxxx

@dmcilvaney dmcilvaney changed the title WIP Better toolchain downloader Jul 26, 2024
@dmcilvaney dmcilvaney force-pushed the damcilva/3.0/tools/better_toolchain_download branch 2 times, most recently from 665a7fd to 0e795d9 Compare July 29, 2024 18:22
@dmcilvaney dmcilvaney marked this pull request as ready for review July 29, 2024 18:37
@dmcilvaney dmcilvaney requested a review from a team as a code owner July 29, 2024 18:37
@dmcilvaney dmcilvaney force-pushed the damcilva/3.0/tools/better_toolchain_download branch from 0e795d9 to ccde108 Compare July 29, 2024 18:39
echo "-u|--url-list: Space separated list of base URLs to attempt to download the RPM from. The URLs will be tried in order"
echo " until the RPM is successfully downloaded. Full URL will be <url>/<rpm_name>"
echo "-c|--certificate: Optional path to a certificate file to use for the download"
echo "-k|--private-key: Optional path to a private key file to use for the download"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does not match usage output above (--key vs --private-key, and --certificate vs --cert)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

--downloader-tool "$(go-downloader)" \
--rpm-name "$(notdir $@)" \
--dst "$@" \
--log-base "$$log_file" \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this intentional? ($$log_file)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, its referencing the bash variable $log_file but needs an escape for Make.

@dmcilvaney dmcilvaney merged commit 150888f into microsoft:3.0-dev Jul 29, 2024
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants