bcachefs-tools: 1.7.0-unstable-2024-05-09 -> 1.9.2

[JohnRTitor]

Description of changes

• Closes Update request: bcachefs-tools 1.7.0-unstable-2024-05-09 → 1.9.1 #321049
• Remove patches that are merged upstream

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

---

Add a 👍 reaction to pull requests you find important.

[tmuehlbacher]

Thanks, but please pause this for a bit, the v1.9.0 tag is a bit messy. The nixpkgs#nixosTests.bcachefs test fails with this.

[JohnRTitor]

Indeed the test fails, here's the log for reference

(finished: must succeed: echo password | bcachefs unlock -k session /dev/vdb1, in 0.70 seconds) machine: must succeed: echo password | mount -t bcachefs /dev/vdb1:/dev/vdb2 /tmp/mnt machine # [ 6.710377] dhcpcd[664]: eth0: Router Advertisement from fe80::2 machine # [ 6.710950] dhcpcd[664]: eth0: adding address fec0::5054:ff:fe12:3456/64 machine # [ 6.712111] dhcpcd[664]: eth0: adding route to fec0::/64 machine # [ 6.712579] dhcpcd[664]: eth0: adding default route via fe80::2 machine: output: ERROR - bcachefs::commands::mount: Fatal error: failed to verify passphrase

cleanup
kill machine (pid 9)
qemu-kvm: terminating on signal 15 from pid 6 (/nix/store/6b1fqdwb3g56j5pazv8zkx9qd0mv3wiz-python3-3.11.9/bin/python3.11)
(finished: cleanup, in 0.02 seconds)
Traceback (most recent call last):
File "/nix/store/s4mmd0j5griaizximy55wdc75gpqiihi-nixos-test-driver-1.1/bin/.nixos-test-driver-wrapped", line 9, in
sys.exit(main())
^^^^^^
File "/nix/store/s4mmd0j5griaizximy55wdc75gpqiihi-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/init.py", line 146, in main
driver.run_tests()
File "/nix/store/s4mmd0j5griaizximy55wdc75gpqiihi-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/driver.py", line 166, in run_tests
self.test_script()
File "/nix/store/s4mmd0j5griaizximy55wdc75gpqiihi-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/driver.py", line 158, in test_script
exec(self.tests, symbols, None)
File "", line 5, in
File "/nix/store/s4mmd0j5griaizximy55wdc75gpqiihi-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/machine.py", line 611, in succeed
raise Exception(f"command {command} failed (exit code {status})")
Exception: command echo password | mount -t bcachefs /dev/vdb1:/dev/vdb2 /tmp/mnt failed (exit code 1)
kill vlan (pid 7)
error: builder for '/nix/store/r9qfv8c94ca52iyx51v99jwd9kl19lc9-vm-test-run-bcachefs.drv' failed with exit code 1;
last 25 log lines:
> machine # [ 6.710377] dhcpcd[664]: eth0: Router Advertisement from fe80::2
> machine # [ 6.710950] dhcpcd[664]: eth0: adding address fec0::5054:ff:fe12:3456/64
> machine # [ 6.712111] dhcpcd[664]: eth0: adding route to fec0::/64
> machine # [ 6.712579] dhcpcd[664]: eth0: adding default route via fe80::2
> machine: output: ERROR - bcachefs::commands::mount: Fatal error: failed to verify passphrase
>
> cleanup
> kill machine (pid 9)
> qemu-kvm: terminating on signal 15 from pid 6 (/nix/store/6b1fqdwb3g56j5pazv8zkx9qd0mv3wiz-python3-3.11.9/bin/python3.11)
> (finished: cleanup, in 0.02 seconds)
> Traceback (most recent call last):
> File "/nix/store/s4mmd0j5griaizximy55wdc75gpqiihi-nixos-test-driver-1.1/bin/.nixos-test-driver-wrapped", line 9, in
> sys.exit(main())
> ^^^^^^
> File "/nix/store/s4mmd0j5griaizximy55wdc75gpqiihi-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/init.py", line 146, in main
> driver.run_tests()
> File "/nix/store/s4mmd0j5griaizximy55wdc75gpqiihi-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/driver.py", line 166, in run_tests
> self.test_script()
> File "/nix/store/s4mmd0j5griaizximy55wdc75gpqiihi-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/driver.py", line 158, in test_script
> exec(self.tests, symbols, None)
> File "", line 5, in
> File "/nix/store/s4mmd0j5griaizximy55wdc75gpqiihi-nixos-test-driver-1.1/lib/python3.11/site-packages/test_driver/machine.py", line 611, in succeed
> raise Exception(f"command {command} failed (exit code {status})")
> Exception: command echo password | mount -t bcachefs /dev/vdb1:/dev/vdb2 /tmp/mnt failed (exit code 1)
> kill vlan (pid 7)
For full logs, run 'nix log /nix/store/r9qfv8c94ca52iyx51v99jwd9kl19lc9-vm-test-run-bcachefs.drv'.

I'll mark it as draft for now, although this works in my unencrypted system, merging this would probably break encrypted or more complex setups.

[tmuehlbacher]

There's likely going to be a v1.9.1 shortly 😅

[tmuehlbacher]

v1.9.1 is tagged now btw. :)

[JohnRTitor]

nom build .#nixosTests.bcachefs succeeds now.

[JohnRTitor]

Looks like it fails multi installer test, @tmuehlbacher would there be a upstream commit fixing this soon?

I remember Kent said this is harmless, ie. won't eat your data. But since this test fails, I am hesitant to merge this as this might cause other issues.

@onny would love if you could test this with encrypted root!

[tmuehlbacher]

simple and multi both fail. Huh, it's stuck at writing the hardware-configuration.nix file? => This is fixed by adding modprobe bcachefs in createPartitions/installer.succeed().

But then it failed for me locally at a later stage verifying the nix store. Not sure about that yet.

I have been using the flake from upstream as an overlay on my system for a bit and can still boot at least.
btw, I think you can also delete the patch file, since it's no longer used?

[JohnRTitor]

Good catch! not sure how I missed it.

[tmuehlbacher]

Ok so I have a fix for these test failures now. The tests work if you add this:

diff --git a/nixos/tests/installer.nix b/nixos/tests/installer.nix
index 3f57a64333dd..d3e870f7922a 100644
--- a/nixos/tests/installer.nix
+++ b/nixos/tests/installer.nix
@@ -1144,6 +1144,7 @@ in {

     createPartitions = ''
       installer.succeed(
+        "modprobe bcachefs",
         "flock /dev/vda parted --script /dev/vda -- mklabel msdos"
         + " mkpart primary ext2 1M 100MB"          # /boot
         + " mkpart primary linux-swap 100M 1024M"  # swap
@@ -1213,6 +1214,7 @@ in {

     createPartitions = ''
       installer.succeed(
+        "modprobe bcachefs",
         "flock /dev/vda parted --script /dev/vda -- mklabel msdos"
         + " mkpart primary ext2 1M 100MB"          # /boot
         + " mkpart primary linux-swap 100M 1024M"  # swap

I am not sure why this is needed now and if that's a problem or normal. Without explicitly loading bcachefs, the mount succeeds still but then the fs cannot be used in any way. It appears that nixos-generate-config just blocks forever trying to write to a file inside of the fs.

---

I was also sidetracked by something else: I have TMPDIR=/var/tmp set for nix-daemon.service because large builds like the kernel end up exhausting the my tmpfs-backed /tmp, which is the default build dir for nix. The problem is that these tests fail if I build them in my bcachefs-backed /var/tmp with kernel messages like this:

installer # [   65.856006] bcachefs (vda3 inum 1879055323 offset 3871232): data data checksum error, type crc32c: got 00000000 should be ee7e0061
installer # [   65.856683] bcachefs (vda3 inum 1879055323 offset 3871232): no device to read from
installer # [   65.857233] bcachefs (vda3 inum 1879055323 offset 3871232): read error 3 from btree lookup
installer # [   65.858518] bcachefs (vda3 inum 1879055323 offset 3871232): data data checksum error, type crc32c: got 00000000 should be ee7e0061
installer # [   65.859271] bcachefs (vda3 inum 1879055323 offset 3871232): no device to read from
installer # [   65.859642] bcachefs (vda3 inum 1879055323 offset 3871232): read error 3 from btree lookup

Here is the full log for # nix build --rebuild --option build-dir /var/tmp -L .#nixosTests.installer.bcachefsSimple
test-build-log.txt

[JohnRTitor]

Modeprobeing loads the kernel module, which I believe takes over mounting. I am not confident enough to merge this as it is, without comprehensive testing that this won't break like last time.

Ideally, issues like these should be fixed upstream, unless we are doing something very wrong with our testing suite.

[JohnRTitor]

@ofborg eval

[JohnRTitor]

Result of nixpkgs-review pr 321055 run on x86_64-linux 1

1 package built:

• bcachefs-tools

[JohnRTitor]

The issue was with bcachefs kernel module, and has been fixed in 6.9.7+ releases.

and Ofborg suggests the installer tests are succeeding as well. As such I have no reason to hold this further.

[toastal]

The issue was with bcachefs kernel module, and has been fixed in 6.9.7+ releases.

Awesome. Can’t wait to give this another try when I travel back home. Good work!