openssh: put tests into passthru

[nikstur]

openssh tests are very slow (they take ~30 mins because they are not parallelized). This makes rebuilding (esp. after changes e.g. to systemd) painful. Putting the tests into a separate derivation solves this without losing any testing capability.

Debian does the same: https://packages.debian.org/sid/openssh-tests

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

---

Add a 👍 reaction to pull requests you find important.

[nikstur]

@ofborg test openssh

[AndersonTorres]

Nice idea!

[JulienMalka]

I don't think Hydra automatically builds passthrough.tests so it will not build the tests as part of building openssh.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-already-reviewed/2617/1346

[risicle]

Suggest using the finalAttrs/finalPackage pattern instead of taking openssh as an argument. This will ensure you're testing the exact variant you think you're testing.

[SuperSandro2000]

I am basically doing the same in an overlay since a longer time because the tests are so annoyingly slow.

[ajs124]

hm. why does this fail on aarch64-linux?

[nikstur]

hm. why does this fail on aarch64-linux?

I think that was just an ofborg runner issue. Seems to be fine now

[drupol]

I don't think Hydra automatically builds passthrough.tests so it will not build the tests as part of building openssh.

@nikstur I really would like to merge this, but this feedback ^^ needs to be addressed.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-already-reviewed/2617/1559

[nikstur]

this feedback ^^ needs to be addressed.

I guess the solution is to explicitly add the openssh tests to a hydra jobset. I don't know much about hydra though. Any pointers would be appreciated.

[nikstur]

this feedback ^^ needs to be addressed.

I added this to release-small now. I would love if someone that knows more about hydra could review this.

[Mic92]

Seems to evaluate on my machine:

nix-eval-jobs -- --force-recurse nixos/release-small.nix  306.57s user 137.58s system 52% cpu 14:04.92 total