Merge staging-next into staging

doc/languages-frameworks/rust.section.md

@@ -13,7 +13,7 @@ into your `configuration.nix` or bring them into scope with `nix-shell -p rustc
 
 For other versions such as daily builds (beta and nightly),
 use either `rustup` from nixpkgs (which will manage the rust installation in your home directory),
-or use Mozilla's [Rust nightlies overlay](#using-the-rust-nightlies-overlay).
+or use a community maintained [Rust overlay](#using-community-rust-overlays).
 
 ## Compiling Rust applications with Cargo {#compiling-rust-applications-with-cargo}
 
@@ -900,76 +900,107 @@ rustc 1.26.0-nightly (188e693b3 2018-03-26)
 
 To see that you are using nightly.
 
-## Using the Rust nightlies overlay {#using-the-rust-nightlies-overlay}
+## Using community Rust overlays {#using-community-rust-overlays}
 
-Mozilla provides an overlay for nixpkgs to bring a nightly version of Rust into scope.
-This overlay can _also_ be used to install recent unstable or stable versions
-of Rust, if desired.
+There are two community maintained approaches to Rust toolchain management:
+- [oxalica's Rust overlay](https://github.com/oxalica/rust-overlay)
+- [fenix](https://github.com/nix-community/fenix)
 
-### Rust overlay installation {#rust-overlay-installation}
-
-You can use this overlay by either changing your local nixpkgs configuration,
-or by adding the overlay declaratively in a nix expression, e.g. in `configuration.nix`.
-For more information see [the manual on installing overlays](#sec-overlays-install).
-
-#### Imperative rust overlay installation {#imperative-rust-overlay-installation}
+Oxalica's overlay allows you to select a particular Rust version and components.
+See [their documentation](https://github.com/oxalica/rust-overlay#rust-overlay) for more
+detailed usage.
 
-Clone [nixpkgs-mozilla](https://github.com/mozilla/nixpkgs-mozilla),
-and create a symbolic link to the file
-[rust-overlay.nix](https://github.com/mozilla/nixpkgs-mozilla/blob/master/rust-overlay.nix)
-in the `~/.config/nixpkgs/overlays` directory.
+Fenix is an alternative to `rustup` and can also be used as an overlay.
 
-```ShellSession
-$ git clone https://github.com/mozilla/nixpkgs-mozilla.git
-$ mkdir -p ~/.config/nixpkgs/overlays
-$ ln -s $(pwd)/nixpkgs-mozilla/rust-overlay.nix ~/.config/nixpkgs/overlays/rust-overlay.nix
-```
+Both Oxalica's overlay and fenix better integrate with nix and cache optimizations.
+Because of this and ergonomics, either of those community projects
+should be preferred to the Mozilla's Rust overlay (nixpkgs-mozilla).
 
-### Declarative rust overlay installation {#declarative-rust-overlay-installation}
-
-Add the following to your `configuration.nix`, `home-configuration.nix`, `shell.nix`, or similar:
+### How to select a specific rustc and toolchain version {#how-to-select-a-specific-rustc-and-toolchain-version}
 
+You can consume the oxalica overlay and use it to grab a specific Rust toolchain version.
+Here is an example `shell.nix` showing how to grab the current stable toolchain:
 ```nix
 { pkgs ? import <nixpkgs> {
  overlays = [
- (import (builtins.fetchTarball https://github.com/mozilla/nixpkgs-mozilla/archive/master.tar.gz))
- # Further overlays go here
+ (import (fetchTarball "https://github.com/oxalica/rust-overlay/archive/master.tar.gz"))
  ];
- };
-};
+ }
+}:
+pkgs.mkShell {
+ nativeBuildInputs = with pkgs; [
+ pkg-config
+ rust-bin.stable.latest.minimal
+ ];
+}
 ```
 
-Note that this will fetch the latest overlay version when rebuilding your system.
+You can try this out by:
+1. Saving that to `shell.nix`
+2. Executing `nix-shell --pure --command 'rustc --version'`
 
-### Rust overlay usage {#rust-overlay-usage}
+As of writing, this prints out `rustc 1.56.0 (09c42c458 2021-10-18)`.
 
-The overlay contains attribute sets corresponding to different versions of the rust toolchain, such as:
+### How to use an overlay toolchain in a derivation {#how-to-use-an-overlay-toolchain-in-a-derivation}
 
-* `latest.rustChannels.stable`
-* `latest.rustChannels.nightly`
-* a function `rustChannelOf`, called as `(rustChannelOf { date = "2018-04-11"; channel = "nightly"; })`, or...
-* `(nixpkgs.rustChannelOf { rustToolchain = ./rust-toolchain; })` if you have a local `rust-toolchain` file (see https://github.com/mozilla/nixpkgs-mozilla#using-in-nix-expressions for an example)
+You can also use an overlay's Rust toolchain with `buildRustPackage`.
+The below snippet demonstrates invoking `buildRustPackage` with an oxalica overlay selected Rust toolchain:
+```nix
+with import <nixpkgs> {
+ overlays = [
+ (import (fetchTarball "https://github.com/oxalica/rust-overlay/archive/master.tar.gz"))
+ ];
+};
 
-Each of these contain packages such as `rust`, which contains your usual rust development tools with the respective toolchain chosen.
-For example, you might want to add `latest.rustChannels.stable.rust` to the list of packages in your configuration.
+rustPlatform.buildRustPackage rec {
+ pname = "ripgrep";
+ version = "12.1.1";
+ nativeBuildInputs = [
+ rust-bin.stable.latest.minimal
+ ];
+
+ src = fetchFromGitHub {
+ owner = "BurntSushi";
+ repo = "ripgrep";
+ rev = version;
+ sha256 = "1hqps7l5qrjh9f914r5i6kmcz6f1yb951nv4lby0cjnp5l253kps";
+ };
 
-Imperatively, the latest stable version can be installed with the following command:
+ cargoSha256 = "03wf9r2csi6jpa7v5sw5lpxkrk4wfzwmzx7k3991q3bdjzcwnnwp";
 
-```ShellSession
-$ nix-env -Ai nixpkgs.latest.rustChannels.stable.rust
+ meta = with lib; {
+ description = "A fast line-oriented regex search tool, similar to ag and ack";
+ homepage = "https://github.com/BurntSushi/ripgrep";
+ license = licenses.unlicense;
+ maintainers = [ maintainers.tailhook ];
+ };
+}
 ```
 
-Or using the attribute with nix-shell:
+Follow the below steps to try that snippet.
+1. create a new directory
+1. save the above snippet as `default.nix` in that directory
+1. cd into that directory and run `nix-build`
 
-```ShellSession
-$ nix-shell -p nixpkgs.latest.rustChannels.stable.rust
-```
+### Rust overlay installation {#rust-overlay-installation}
+
+You can use this overlay by either changing your local nixpkgs configuration,
+or by adding the overlay declaratively in a nix expression, e.g. in `configuration.nix`.
+For more information see [the manual on installing overlays](#sec-overlays-install).
+
+### Declarative Rust overlay installation {#declarative-rust-overlay-installation}
 
-Substitute the `nixpkgs` prefix with `nixos` on NixOS.
-To install the beta or nightly channel, "stable" should be substituted by
-"nightly" or "beta", or
-use the function provided by this overlay to pull a version based on a
-build date.
+This snippet shows how to use oxalica's Rust overlay.
+Add the following to your `configuration.nix`, `home-configuration.nix`, `shell.nix`, or similar:
 
-The overlay automatically updates itself as it uses the same source as
-[rustup](https://www.rustup.rs/).
+```nix
+{ pkgs ? import <nixpkgs> {
+ overlays = [
+ (import (builtins.fetchTarball "https://github.com/oxalica/rust-overlay/archive/master.tar.gz"))
+ # Further overlays go here
+ ];
+ };
+};
+```
+
+Note that this will fetch the latest overlay version when rebuilding your system.

maintainers/maintainer-list.nix

@@ -6252,6 +6252,12 @@
  githubId = 278013;
  name = "Tomasz Kontusz";
  };
+ kubukoz = {
+ email = "[email protected]";
+ github = "kubukoz";
+ githubId = 894884;
+ name = "Jakub Kozłowski";
+ };
  kurnevsky = {
  email = "[email protected]";
  github = "kurnevsky";

maintainers/scripts/find-tarballs.nix

@@ -37,7 +37,7 @@ let
  keyDrv = drv: if canEval drv.drvPath then { key = drv.drvPath; value = drv; } else { };
 
  immediateDependenciesOf = drv:
- concatLists (mapAttrsToList (n: v: derivationsIn v) (removeAttrs drv ["meta" "passthru"]));
+ concatLists (mapAttrsToList (n: v: derivationsIn v) (removeAttrs drv (["meta" "passthru"] ++ optionals (drv?passthru) (attrNames drv.passthru))));
 
  derivationsIn = x:
  if !canEval x then []

nixos/doc/manual/from_md/release-notes/rl-2111.section.xml

@@ -1754,6 +1754,14 @@ Superuser created successfully.
  better user experience and benefit from this change.
  </para>
  </listitem>
+ <listitem>
+ <para>
+ A new option
+ <literal>services.prometheus.enableReload</literal> has been
+ added which can be enabled to reload the prometheus service
+ when its config file changes instead of restarting.
+ </para>
+ </listitem>
  <listitem>
  <para>
  Dokuwiki now supports caddy! However

nixos/doc/manual/release-notes/rl-2111.section.md

@@ -497,6 +497,8 @@ In addition to numerous new and upgraded packages, this release has the followin
 
 - The `cawbird` Twitter client now uses its own API keys to count as different application than upstream builds. This is done to evade application-level rate limiting. While existing accounts continue to work, users may want to remove and re-register their account in the client to enjoy a better user experience and benefit from this change.
 
+- A new option `services.prometheus.enableReload` has been added which can be enabled to reload the prometheus service when its config file changes instead of restarting.
+
 - Dokuwiki now supports caddy! However
  - the nginx option has been removed, in the new configuration, please use the `dokuwiki.webserver = "nginx"` instead.
  - The "${hostname}" option has been deprecated, please use `dokuwiki.sites = [ "${hostname}" ]` instead

nixos/modules/services/monitoring/prometheus/default.nix

@@ -7,6 +7,30 @@ let
 
  workingDir = "/var/lib/" + cfg.stateDir;
 
+ prometheusYmlOut = "${workingDir}/prometheus-substituted.yaml";
+
+ writeConfig = pkgs.writeShellScriptBin "write-prometheus-config" ''
+ PATH="${makeBinPath (with pkgs; [ coreutils envsubst ])}"
+ touch '${prometheusYmlOut}'
+ chmod 600 '${prometheusYmlOut}'
+ envsubst -o '${prometheusYmlOut}' -i '${prometheusYml}'
+ '';
+
+ triggerReload = pkgs.writeShellScriptBin "trigger-reload-prometheus" ''
+ PATH="${makeBinPath (with pkgs; [ systemd ])}"
+ if systemctl -q is-active prometheus.service; then
+ systemctl reload prometheus.service
+ fi
+ '';
+
+ reload = pkgs.writeShellScriptBin "reload-prometheus" ''
+ PATH="${makeBinPath (with pkgs; [ systemd coreutils gnugrep ])}"
+ cursor=$(journalctl --show-cursor -n0 | grep -oP "cursor: \K.*")
+ kill -HUP $MAINPID
+ journalctl -u prometheus.service --after-cursor="$cursor" -f \
+ | grep -m 1 "Completed loading of configuration file" > /dev/null
+ '';
+
  # a wrapper that verifies that the configuration is valid
  promtoolCheck = what: name: file:
  if cfg.checkConfig then
@@ -47,7 +71,11 @@ let
 
  cmdlineArgs = cfg.extraFlags ++ [
  "--storage.tsdb.path=${workingDir}/data/"
- "--config.file=/run/prometheus/prometheus-substituted.yaml"
+ "--config.file=${
+ if cfg.enableReload
+ then prometheusYmlOut
+ else "/run/prometheus/prometheus-substituted.yaml"
+ }"
  "--web.listen-address=${cfg.listenAddress}:${builtins.toString cfg.port}"
  "--alertmanager.notification-queue-capacity=${toString cfg.alertmanagerNotificationQueueCapacity}"
  "--alertmanager.timeout=${toString cfg.alertmanagerTimeout}s"
@@ -731,6 +759,25 @@ in {
  '';
  };
 
+ enableReload = mkOption {
+ default = false;
+ type = types.bool;
+ description = ''
+ Reload prometheus when configuration file changes (instead of restart).
+
+ The following property holds: switching to a configuration
+ (<literal>switch-to-configuration</literal>) that changes the prometheus
+ configuration only finishes successully when prometheus has finished
+ loading the new configuration.
+
+ Note that prometheus will also get reloaded when the location of the
+ <option>environmentFile</option> changes but not when its contents
+ changes. So when you change it contents make sure to reload prometheus
+ manually or include the hash of <option>environmentFile</option> in its
+ name.
+ '';
+ };
+
  environmentFile = mkOption {
  type = types.nullOr types.path;
  default = null;
@@ -928,23 +975,67 @@ in {
  systemd.services.prometheus = {
  wantedBy = [ "multi-user.target" ];
  after = [ "network.target" ];
- preStart = ''
+ preStart = mkIf (!cfg.enableReload) ''
  ${lib.getBin pkgs.envsubst}/bin/envsubst -o "/run/prometheus/prometheus-substituted.yaml" \
  -i "${prometheusYml}"
  '';
  serviceConfig = {
  ExecStart = "${cfg.package}/bin/prometheus" +
  optionalString (length cmdlineArgs != 0) (" \\\n " +
  concatStringsSep " \\\n " cmdlineArgs);
+ ExecReload = mkIf cfg.enableReload "+${reload}/bin/reload-prometheus";
  User = "prometheus";
  Restart = "always";
- EnvironmentFile = mkIf (cfg.environmentFile != null) [ cfg.environmentFile ];
+ EnvironmentFile = mkIf (cfg.environmentFile != null && !cfg.enableReload) [ cfg.environmentFile ];
  RuntimeDirectory = "prometheus";
  RuntimeDirectoryMode = "0700";
  WorkingDirectory = workingDir;
  StateDirectory = cfg.stateDir;
  StateDirectoryMode = "0700";
  };
  };
+ systemd.services.prometheus-config-write = mkIf cfg.enableReload {
+ wantedBy = [ "prometheus.service" ];
+ before = [ "prometheus.service" ];
+ serviceConfig = {
+ Type = "oneshot";
+ User = "prometheus";
+ StateDirectory = cfg.stateDir;
+ StateDirectoryMode = "0700";
+ EnvironmentFile = mkIf (cfg.environmentFile != null) [ cfg.environmentFile ];
+ ExecStart = "${writeConfig}/bin/write-prometheus-config";
+ };
+ };
+ # prometheus-config-reload will activate after prometheus. However, what we
+ # don't want is that on startup it immediately reloads prometheus because
+ # prometheus itself might have just started.
+ #
+ # Instead we only want to reload prometheus when the config file has
+ # changed. So on startup prometheus-config-reload will just output a
+ # harmless message and then stay active (RemainAfterExit).
+ #
+ # Then, when the config file has changed, switch-to-configuration notices
+ # that this service has changed and needs to be reloaded
+ # (reloadIfChanged). The reload command then actually writes the new config
+ # and reloads prometheus.
+ systemd.services.prometheus-config-reload = mkIf cfg.enableReload {
+ wantedBy = [ "prometheus.service" ];
+ after = [ "prometheus.service" ];
+ reloadIfChanged = true;
+ serviceConfig = {
+ Type = "oneshot";
+ User = "prometheus";
+ StateDirectory = cfg.stateDir;
+ StateDirectoryMode = "0700";
+ EnvironmentFile = mkIf (cfg.environmentFile != null) [ cfg.environmentFile ];
+ RemainAfterExit = true;
+ TimeoutSec = 60;
+ ExecStart = "${pkgs.logger}/bin/logger 'prometheus-config-reload will only reload prometheus when reloaded itself.'";
+ ExecReload = [
+ "${writeConfig}/bin/write-prometheus-config"
+ "+${triggerReload}/bin/trigger-reload-prometheus"
+ ];
+ };
+ };
  };
 }

nixos/modules/system/etc/setup-etc.pl

@@ -138,3 +138,9 @@ sub link {
 # Rewrite /etc/.clean.
 close CLEAN;
 write_file("/etc/.clean", map { "$_\n" } @copied);
+
+# Create /etc/NIXOS tag if not exists.
+# When /etc is not on a persistent filesystem, it will be wiped after reboot,
+# so we need to check and re-create it during activation.
+open TAG, ">>/etc/NIXOS";
+close TAG;