Nix flake

[ysthakur]

Closes #10352

Description

This PR adds a flake.nix file so that Nushell can be available as a Nix flake.

One problem with this is that because the rust toolchain is gotten from rust-toolchain.toml, its hash must be manually entered into flake.nix every time the toolchain is updated. However, the toolchain isn't updated super frequently, so that might be acceptable.

User-Facing Changes

Nix users will be able to use the github:nushell/nushell flake.

Tests + Formatting

I'm not sure how to test this but I'll get around to it.

After Submitting

TODO update documentation

[fdncred]

This was tried before #8155. I personally don't have a problem with it but other maintainers may.

[ysthakur]

Whoops, I didn't see that PR. I guess the concerns about CI still apply and MangoIV's flake works already (and is much more comprehensive)

[poelzi]

I guess it will become the new normal for us nix users to create flake PRs just to find out, somebody already did it and it did not get commited. So sad the world is stuck in the last century sometimes

[crabdancing]

@poelzi Yeah, that's a mood. I wanted to quickly have my nix toolchain build & run the latest master branch just so I could try to reproduce some issues I saw, only to realize I'd have to rely on overlaying nixpkgs or using some less robust build system -- and I decided to not bother. :P

I actually do really wonder how many Nushell users happen to also use Nix, as both are massive paradigm shifts in conventional ways of doing things and could be classed as 'weird ML things'.

[fdncred]

We are open to having a nix flake here https://github.com/nushell/integrations

[crabdancing]

Oh! Well, I'd be willing to PR it there, but because of the way Nix flakes work, you'd have to bump the lock file on the integrations repo as this repo updates -- otherwise it'll keep reproducing earlier versions of the nushell repo instead of current.

This is because Nix flakes put a lot of emphasis on reproducibility and purity (minimizing wherever possible the outside changes that can affect the output of computations). It's great for being confident that something that works today can be made to work in 6 months or 10 years, even on a different system -- but it's not great for splitting up the flake into a separate repo, since the flake.lock must be in version tracking somewhere, and has to be updated if you actually want the latest version.

That said, it's totally possible to automate flake.lock bumping with CI, and it is pretty reliable in my experience. How do you feel about that?

[fdncred]

I doubt any core-team member would be willing to do this manually so if there's a fool-proof automated way to do it in the integrations repo, then we could try that.

[crabdancing]

Honstly I'm not sure if it's worth it, if it can't go in the main repo -- it would cause a lot of bloat to the commit history of this repo, and add an extra point of failure. (E.g., if the main repo's path changes, or if CI automation breaks, or if MR key environment changes somehow).

It also doesn't inherit some of the other advantages, such as easily building and running forks of the main repo to quickly test out features. That advantage comes from flakes being tightly coupled with VCS, and each fork inheriting a flake.nix that specifies a 'ground truth' for the nix build process of a given project commit.

So, for example, if a project is 'flakified', I can do something like run

nix run github:nushell/nushell

to automatically build & run the latest unstable, or I can do

nix run github:nushell/nushell/db86dd9f267e46a7a8aeff75ab87b3d0832d1e00

to build & run a specific commit of nushell, or I can build & run a specific PR by doing

nix run github:somerandomperson/nushell/feat-add-ponies

to try out their feature branch and see if it works.

The really nice thing about it is someone doesn't need to know anything about cargo or rustc to be able to do this reliably, nor do they need to worry about pulling in C dependencies and making sure they're present. Being able to abstract away all of that into a command that just says 'run this version of this thing, and if you don't have it -- build it, using exactly these versions of all these dependencies' in a secure, reliable, cross-language, and reproducible way is game changing.

But a lot of that doesn't work if the flake is separate, because then the commit hashes for a given nushell repo commit aren't the same as the commit hashes for integrations, and the situation gets even worse if you're trying to quickly build & run a random fork. It's still possible, but it's annoying -- to the point that I end up not testing new Nushell features and not bothering up to update my system flake to the latest version of Nushell, and that's kind of sad because I'd be happy to contribute more to development if it weren't so inconvenient -- Nushell is probably my favorite project on Github.

It's also kind of sad that so many Nix build environments use Bash, and it would be awesome if Nushell got more traction in the Nix ecosystem -- there have already been experiments with using Nushell as a replacement.

I'm not sure why they're so resistant to putting it in the main repo -- I guess it just feels like bloat to them, because they 'already have CI'? I think there's a lot to gain from it, but I guess I can understand the trepidation. Regardless, putting it in an external official git repo is barely better than just using an unofficial flake TBH -- so I don't know if it's worth the time.

[fdncred]

The main issue has always been that it requires the files to be in the root of the repo cluttering up the root folder.

[crabdancing]

Hmmm. I wonder if I could talk to the people making the implementation standard and see if they're open to having an alternate path. Am I right in inferring that having a single directory is acceptable (e.g., nix in the git root) but having two is not? (flake.lock, flake.nix) Or was it the concern that they wanted to add more than two files?

[fdncred]

if it could go in our scripts folder then no one would care because it's out of site.

[crabdancing]

It looks like they also wanted to add shell.nix and default.nix. These do not have to be in the root directory, and in fact, don't have to be present at all. They are legacy niceties in the Nix world, and you can embed all the logic surrounding the imperative development environment in devShells, in the flake itself to minimize clutter if desirable.

If there was a PR that only added two files to root, and didn't touch anything else, and was confirmed to result in a perfect reproducible build of Nushell, would that be acceptable?

[crabdancing]

Actually, apparently it does not even have to be in the root! I didn't know about this myself, but Nix already supports syntax to access a flake.nix in arbitrary locations of the git repo.

So no modification of upstream logic would even be needed on the Nix side to reduce clutter. It would inconvenience Nix users a lil bit, but it's certainly an improvement if they can build from the official repo!

Should I go ahead and open an issue about this?

[fdncred]

sure. if you put it in the scripts folder with nothing extra in the root of the repo, i'll land it.