build: pack checksums into fewer files, by target #38963
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
vtjnash
added
domain:building
Hopefully this should preserve the ability to generate these in parallel (and keep a clean tree) and prevent conflicts when updating different deps targets, while consolidating any related files
Disables progress bars, which are often illegible anyways with parallel make invocations.
vtjnash
force-pushed
the
jn/checksum-packs
branch
from
a624753
to
75be274
Compare
staticfloat
requested changes
Dec 23, 2020
| # Add this guy to his project target (e.g. `make -f contrib/refresh_checksums.mk openblas`) | ||
| $(1): checksum-$(1)-$(2)-$(3) | ||
| # Add a dependency to the pack target | ||
| # TODO: can we make this so it only adds an ordering but not a dependency? |
There was a problem hiding this comment.
I believe this TODO has been satisfied.
There was a problem hiding this comment.
any suggestion on how to do it?
| CURL_OPTS="-fkL --connect-timeout $TIMEOUT -y $TIMEOUT" | ||
| FETCH_OPTS="-T $TIMEOUT" | ||
| WGET_OPTS="-nv --no-check-certificate --no-use-server-timestamps --tries=1 --timeout=$TIMEOUT" | ||
| CURL_OPTS="-fkLSs --connect-timeout $TIMEOUT -y $TIMEOUT" |
There was a problem hiding this comment.
I'm 70% against making these completely silent; on slow connections it may look like the build is needlessly frozen.
There was a problem hiding this comment.
we already silence much of what happens in deps. I contemplated making it conditional on the presence of --jobserver-fds= (or -j) in MAKEFLAGS, but didn't seem worth it
There was a problem hiding this comment.
Most of what we silence doesn't depend on network speed though; A users build could potentially freeze for 2-3 minutes due to being stuck downloading an LLVM build; I think we should continue to print out download status reports to stdout.
staticfloat
requested changes
Dec 26, 2020
vtjnash
added a commit
that referenced
this pull request
Dec 26, 2020
Hopefully this should preserve the ability to generate these in parallel (and keep a clean tree) and prevent conflicts when updating different deps targets, while consolidating any related files
|
Merged in 8af7a04 |
KristofferC
pushed a commit
that referenced
this pull request
Dec 27, 2020
Hopefully this should preserve the ability to generate these in parallel (and keep a clean tree) and prevent conflicts when updating different deps targets, while consolidating any related files (cherry picked from commit 8af7a04)
staticfloat
pushed a commit
that referenced
this pull request
Jan 15, 2021
Hopefully this should preserve the ability to generate these in parallel (and keep a clean tree) and prevent conflicts when updating different deps targets, while consolidating any related files (cherry picked from commit 8af7a04)
ElOceanografo
pushed a commit
to ElOceanografo/julia
that referenced
this pull request
May 4, 2021
Hopefully this should preserve the ability to generate these in parallel (and keep a clean tree) and prevent conflicts when updating different deps targets, while consolidating any related files
staticfloat
pushed a commit
that referenced
this pull request
Dec 23, 2022
Hopefully this should preserve the ability to generate these in parallel (and keep a clean tree) and prevent conflicts when updating different deps targets, while consolidating any related files (cherry picked from commit 8af7a04)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hopefully this should preserve the ability to generate these in parallel
(and keep a clean tree) and prevent conflicts when updating different
deps targets, while consolidating any related files.
This reduces from ~1000 files to ~40, and makes it feel like this is one
of my more impressive diffs. 😂