Clean up cross bootstrapping #320852
Merged
Clean up cross bootstrapping #320852
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This is the most upstream one, and so to avoid infinite recursion we should get the things from it. This isn't needed per-se now, but will be after the next commit.
8528178
to
b359fe7
Compare
For a long time, we've had `crossLibcStdenv`, `*Cross` libc attributes,
and `*bsdCross` pre-libc package sets. This was always bad because
having "cross" things is "not declarative": the naming doesn't reflect
what packages *need* but rather how we *provide* something. This is
ugly, and creates needless friction between cross and native building.
Now, almost all of these `*Cross` attributes are gone: just these are
kept:
- Glibc's and Musl's are kept, because those packages are widely used
and I didn't want to risk changing the native builds of those at this
time.
- generic `libcCross`, `theadsCross`, and friends, because these relate
to the convolulted GCC bootstrap which still needs to be redone.
The BSD and obscure Linux or freestnanding libcs have conversely all
been made to use a new `stdenvNoLibc`, which is like the old
`crossLibcStdenv` except:
1. It usable for native and cross alike
2. It named according to what it *is* ("a standard environment without
libc but with a C compiler"), rather than some non-compositional
jargon ("the stdenv used for building libc when cross compiling",
yuck).
I should have done this change long ago, but I was stymied because of
"infinite recursions". The problem was that in too many cases we are
overriding `stdenv` to *remove* things we don't need, and this risks
cyles since those more minimal stdenvs are used to build things in the
more maximal stdenvs.
The solution is to pass `stage.nix` `stdenvNoCC`, so we can override to
*build up* rather than *tear down*. For now, the full `stdenv` is also
passed, so I don't need to change the native bootstraps, but I can see
this changing as we make things more uniform and clean those up.
Finally, the BSDs also had to be cleaned up, since they have a few
pre-libc dependencies, demanding a systematic approach. I realized what
rhelmot did in 6120256 (specify what
packages just need `stdenvNoLibc`) is definitely the right approach for
this, and adjusted NetBSD and OpenBSD to likewise use it.
b359fe7
to
51f1eca
Compare
rhelmot
reviewed
Jun 19, 2024
| @@ -91,6 +88,9 @@ lib.makeOverridable ( | |||
| // lib.optionalAttrs stdenv'.hasCC { | |||
| # TODO should CC wrapper set this? | |||
| CPP = "${stdenv'.cc.targetPrefix}cpp"; | |||
|
|
|||
| # Since STRIP below is the flag | |||
There was a problem hiding this comment.
The "below" here refers to makeFlags, should be changed now that it's not adjacent
| @@ -81,6 +85,9 @@ lib.makeOverridable ( | |||
| // lib.optionalAttrs stdenv'.hasCC { | |||
| # TODO should CC wrapper set this? | |||
| CPP = "${stdenv'.cc.targetPrefix}cpp"; | |||
|
|
|||
| # Since STRIP below is the flag | |||
| extraNativeBuildInputs = old.extraNativeBuildInputs | ||
| ++ lib.optionals | ||
| (hostPlatform.isLinux && !buildPlatform.isLinux) | ||
| [ buildPackages.patchelf ] |
There was a problem hiding this comment.
What's up with this? I know it's just being moved, but who's asking for patchelf on literally everything?
There was a problem hiding this comment.
I suppose glass houses and so forth
|
In the interest of not waiting a long time for ofborg again, I will move those comments in a second PR. |
|
@Ericson2314: how did you miss the ofBorg label that it rebuilds a stdenv? |
This one rebuilds both *-darwin stdenvs. Let me revert the PR; you can redo it later. |
vcunat
added a commit
that referenced
this pull request
Jun 21, 2024
|
I am sorry @vcunat. I do not know how I missed that; there is no excuse. |
This was referenced Jun 21, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Contains #320840, will recompute diff once that one is merged.
This will just be one commit then.
Description of changes
For a long time, we've had
crossLibcStdenv,*Crosslibc attributes, and*bsdCrosspre-libc package sets. This was always bad because having "cross" things is "not declarative": the naming doesn't reflect what packages need but rather how we provide something. This is ugly, and creates needless friction between cross and native building.Now, almost all of these
*Crossattributes are gone: just these are kept:Glibc's and Musl's are kept, because those packages are widely used and I didn't want to risk changing the native builds of those at this time.
generic
libcCross,theadsCross, and friends, because these relate to the convolulted GCC bootstrap which still needs to be redone.The BSD and obscure Linux or freestnanding libcs have conversely all been made to use a new
stdenvNoLibc, which is like the oldcrossLibcStdenvexcept:It usable for native and cross alike
It named according to what it is ("a standard environment without libc but with a C compiler"), rather than some non-compositional jargon ("the stdenv used for building libc when cross compiling", yuck).
I should have done this change long ago, but I was stymied because of "infinite recursions". The problem was that in too many cases we are overriding
stdenvto remove things we don't need, and this risks cyles since those more minimal stdenvs are used to build things in the more maximal stdenvs.The solution is to pass
stage.nixstdenvNoCC, so we can override to build up rather than tear down. For now, the fullstdenvis also passed, so I don't need to change the native bootstraps, but I can see this changing as we make things more uniform and clean those up.Finally, the BSDs also had to be cleaned up, since they have a few pre-libc dependencies, demanding a systematic approach. I realized what @rhelmot did in 6120256 (specify what packages just need
stdenvNoLibc) is definitely the right approach for this, and adjusted NetBSD and OpenBSD to likewise use it.Things done
nix.conf? (See Nix manual)sandbox = relaxedsandbox = truenix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage./result/bin/)Add a 👍 reaction to pull requests you find important.