-
-
Notifications
You must be signed in to change notification settings - Fork 13.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Hostname being set too late during bootup causes mismatching log metadata. #318907
Comments
Me neither. I don't see a reason not to have that file. @dasJ may be more knowledgeable, but I'd say go for it. |
I think the main reason was to have the initrds be more substitutable between different hosts as this would be a difference between all of them (when having different systems with only different hostnames). I'm not sure how well adding the hostname file would work. My memory is a bit fuzzy on the details (I did a lot of other projects at that time) but I think an issue was that when journald starts up, it imports the kernel logs BEFORE the hostname was set (causing the pre-systemd lines to be mapped to localhost). An alternative implementation that would probably solve that as well (if it's still/even an issue) would be adding the |
If I read the code correctly, journald reads the hostname from /proc: So the hostname needs to be set system-wide before journald starts up. Systemd does this in So on bare metal we can either set |
It seems this is the best choice then, but maybe also
as that's more standard and picked up even earlier in the boot process. Unless you're running an old kernel though. (<5.19?) |
I'll run an experiment to see if On that note, I also realized there's an equivalent (or slightly harder?) problem with |
(So, setting /etc/hostname in the initrd works. Every message from that boot has the correct hostname.) I was also thinking about your folks' usecase some more, and was wondering about one thing ... how are you approaching the UKI/secureboot? None of the mentioned approaches work for that, right? |
Why not? Most people use https://github.com/nix-community/lanzaboote for that which signs whatever UKI you have locally with a local key |
Because changing files or the kernel cmdline will cause different images to be created? Your design shares a single one among multiple hosts, no? |
Yes, that's the downside. Although, the cmdline is only baked into the UKI which is not in the nix store anyway. This way we could substitute the initrd and have differing hostnames. I must mention, however, that I personally don't care about the substitutability of the initrd. My main focus was to keep the state as it was before, but if people decide it's time to change that, I'm not opposed to that ;) |
With the the Systemd-based initrd, systemd-journald is doing the logging. One of Journald's Trusted Journal Fields is `_HOSTNAME` (systemd.journal-fields(7)). Without explicitly setting the hostname via this file or the kernel cmdline, `localhost` is used and captured in the journal. As a result, a boot's log references multiple hostnames. With centralized log collection this breaks filtering (more so when logs from multiple Systemd-based initrds are streaming in simultaneously. Fixes NixOS#318907.
Describe the bug
Hostname being unset (resulting in
localhost
) causes issues for logging. Specifically, with journald, the same boot (identified by_BOOT_ID
(see systemd.journal-fields(7)) has both_HOSTNAME=localhost
and_HOSTNAME=<intended hostname>
with the switch happening at the rootfs pivot of the boot.Why does this matter?
This makes filtering by hostname difficult. Even more so, when doing log aggregation from multiple machines, filtering by hostname will yield incomplete results. Instead one must build a pipeline that makes looks up all boot_ids of a hostname and querying all those instead.
Steps To Reproduce
Steps to reproduce the behavior:
journalctl -o cat --output-fields=_BOOT_ID,_HOSTNAME -b
Expected behavior
journal contains the the same
_HOSTNAME
as long as it not changed by the user.Hostname should be generally available, even during early boot(?).
Screenshots
N/A
Additional context
I'm filing this as a bug instead of a pull request because I'm not sure if there are any unintended consequences of doing this. Aside from the usual https://xkcd.com/1172/. Hoping for your input here.
There's a reference in the docs giving an example of setting the hostname by hand:
nixpkgs/nixos/modules/system/boot/systemd/initrd.nix
Line 163 in e2dd4e1
Notify maintainers
CC/FYI @dasJ , @roberth who added the example/was tagged in the commit.
Metadata
Please run
nix-shell -p nix-info --run "nix-info -m"
and paste the result.Add a 👍 reaction to issues you find important.
The text was updated successfully, but these errors were encountered: