Rust YJIT build failure "Segmentation fault" | Debian 12 Bookworm - aarch64 (Raspberry pi 3)

[arnlen]

Context

What happens

When trying to build Ruby from sources (via rbenv install 3.3.3) on the latest Raspberry Pi OS (Debian Bookworm port):

• with YJIT (default option), it fails. ❌
• without YJIT (RUBY_CONFIGURE_OPTS="--disable-yjit"), it succeeds. ✅

What I tried

The followings doesn't resolve the issue:

• Increasing swap up to 4GB (via dphys-swapfile)
• Compiling with make option "-j2" (MAKE_OPTS="-j2")
• try another Ruby version: same result for Ruby version 3.1.0, 3.2.0, 3.3.0, 3.3.3

My setup

• Hardware: Raspberry Pi 3 B+
• Memory: 907 MB RAM, 4 GB swap
• OS: Raspberry Pi OS Lite (without desktop) - Debian 12 Bookworm

Versions

• rbenv 1.2.0-91-gc3ba994
• ruby-build 20240612
• rustc 1.63.0
• gcc (Debian 12.2.0-14) 12.2.0

To reproduce

• Install Raspberry Pi OS Bookworm
• Install build dependencies (list here)
• Install Rbenv and Ruby-build

curl -fsSL https://github.com/rbenv/rbenv-installer/raw/HEAD/bin/rbenv-installer | bash

• Install Ruby:

rbenv install 3.3.3 --verbose

• It fails after ≈ 13 minutes on my Raspberry Pi 3

Logs

$ rbenv install 3.3.0 --verbose

==> Installing ruby-3.3.0...

[...]

checking for rustc... rustc
checking whether rustc works for YJIT... yes

[...]

---
Configuration summary for ruby version 3.3.0

   * Installation prefix: [...]/.rbenv/versions/3.3.0
   * exec prefix:         ${prefix}
   * arch:                aarch64-linux
   * site arch:           ${arch}
   * RUBY_BASE_NAME:      ruby
   * enable shared:       yes
   * ruby lib prefix:     ${libdir}/${RUBY_BASE_NAME}
   * site libraries path: ${rubylibprefix}/${sitearch}
   * vendor path:         ${rubylibprefix}/vendor_ruby
   * target OS:           linux
   * compiler:            gcc
   * with thread:         pthread
   * with coroutine:      arm64
   * enable shared libs:  yes
   * dynamic library ext: so
   * CFLAGS:              ${optflags} ${debugflags} ${warnflags}
   * LDFLAGS:             -L. -fstack-protector-strong -rdynamic -Wl,-export-dynamic \
                          -Wl,--no-as-needed
   * DLDFLAGS:            -Wl,--compress-debug-sections=zlib
   * optflags:            -O3 -fno-fast-math
   * debugflags:          -ggdb3
   * warnflags:           -Wall -Wextra -Wdeprecated-declarations -Wdiv-by-zero \
                          -Wduplicated-cond -Wimplicit-function-declaration -Wimplicit-int \
                          -Wpointer-arith -Wwrite-strings -Wold-style-definition \
                          -Wimplicit-fallthrough=0 -Wmissing-noreturn -Wno-cast-function-type \
                          -Wno-constant-logical-operand -Wno-long-long \
                          -Wno-missing-field-initializers -Wno-overlength-strings \
                          -Wno-packed-bitfield-compat -Wno-parentheses-equality -Wno-self-assign \
                          -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value \
                          -Wsuggest-attribute=format -Wsuggest-attribute=noreturn \
                          -Wunused-variable -Wmisleading-indentation -Wundef
   * strip command:       strip -S -x
   * install doc:         rdoc
   * YJIT support:        yes
   * RJIT support:        yes
   * man page type:       doc

---
-> make -j 4
	BASERUBY = ./tool/missing-baseruby.bat
	CC = gcc
	LD = ld
	LDSHARED = gcc -shared
	CFLAGS = -O3 -fno-fast-math -ggdb3 -Wall -Wextra -Wdeprecated-declarations -Wdiv-by-zero -Wduplicated-cond -Wimplicit-function-declaration -Wimplicit-int -Wpointer-arith -Wwrite-strings -Wold-style-definition -Wimplicit-fallthrough=0 -Wmissing-noreturn -Wno-cast-function-type -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-packed-bitfield-compat -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wsuggest-attribute=format -Wsuggest-attribute=noreturn -Wunused-variable -Wmisleading-indentation -Wundef   -fPIC
	XCFLAGS = -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -mbranch-protection=pac-ret -fno-strict-overflow -fvisibility=hidden -fexcess-precision=standard -DRUBY_EXPORT -I. -I.ext/include/aarch64-linux -I./include -I. -I./prism -I./enc/unicode/15.0.0
	CPPFLAGS =
	DLDFLAGS = -Wl,--compress-debug-sections=zlib -Wl,-soname,libruby.so.3.3  -fstack-protector-strong
	SOLIBS = -lz -lrt -lrt -lgmp -ldl -lcrypt -lm -lpthread
	LANG = en_GB.UTF-8
	LC_ALL =
	LC_CTYPE =
	MFLAGS = -j4 --jobserver-auth=6,7
	RUSTC = rustc
	YJIT_RUSTC_ARGS = --crate-name=yjit --crate-type=staticlib --edition=2021 -g -C lto=thin -C opt-level=3 -C overflow-checks=on '--out-dir=/tmp/ruby-build.20240626232448.59556.GhdrJm/ruby-3.3.0/yjit/target/release/' ./yjit/src/lib.rs
gcc (Debian 12.2.0-14) 12.2.0
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

compiling ./main.c
compiling dmydln.c
compiling miniinit.c
compiling dmyext.c

[...]

compiling vm_backtrace.c
compiling vm_dump.c
compiling vm_sync.c
compiling vm_trace.c
compiling weakmap.c
compiling yjit.c

building Rust YJIT (release mode)     # <--------- Rust YJIT build starts here
compiling ./missing/setproctitle.c
compiling ./missing/strlcat.c
compiling ./missing/strlcpy.c
compiling addr2line.c
compiling dmyenc.c
compiling dln.c
compiling localeinit.c
compiling array.c
compiling prism/api_node.c
compiling prism/api_pack.c
compiling prism/diagnostic.c
compiling prism/encoding.c
compiling prism/extension.c
compiling prism/node.c
compiling prism/options.c
compiling prism/pack.c
compiling prism/prettyprint.c
compiling prism/regexp.c
compiling prism/serialize.c
compiling prism/token_type.c
compiling prism/util/pm_buffer.c

make: *** [yjit/yjit.mk:26: yjit/target/release/libyjit.a] Segmentation fault
make: *** Waiting for unfinished jobs....
external command failed with status 2

BUILD FAILED (Debian 12 on aarch64 using ruby-build 20240612)

Alternative logs

I notice that, when building with the following options, the failure occurs a bit earlier (when compiling prism/serialize.c rather than when compiling prism/util/pm_buffer.c) and the output of the compilation of prism/regexp.c is also different:

CONFIGURE_OPTS="--disable-install-doc --enable-shared" rbenv install 3.3.0 --verbose

[...]

compiling weakmap.c
compiling yjit.c

building Rust YJIT (release mode)
compiling ./missing/setproctitle.c
compiling ./missing/strlcat.c
compiling ./missing/strlcpy.c
compiling addr2line.c
compiling dmyenc.c
compiling dln.c
compiling localeinit.c
compiling array.c
compiling prism/api_node.c
compiling prism/api_pack.c
compiling prism/diagnostic.c
compiling prism/encoding.c
compiling prism/extension.c
compiling prism/node.c
compiling prism/options.c
compiling prism/pack.c
compiling prism/prettyprint.c
compiling prism/regexp.c
/lib/aarch64-linux-gnu/librustc_driver-fe2d3f0c4114cdf4.so(+0x6c4430)[0x7f8af84430]
linux-vdso.so.1(__kernel_rt_sigreturn+0x0)[0x7f8dff87b0]
/lib/aarch64-linux-gnu/libLLVM-14.so.1(+0x132a9f0)[0x7f855fa9f0]
/lib/aarch64-linux-gnu/libLLVM-14.so.1(_ZN4llvm11raw_ostream5writeEPKcm+0x164)[0x7f8514b49c]
/lib/aarch64-linux-gnu/libLLVM-14.so.1(_ZN4llvm9MCContext16createTempSymbolERKNS_5TwineEb+0xb4)[0x7f86501048]
/lib/aarch64-linux-gnu/libLLVM-14.so.1(_ZN4llvm9MCContext16createTempSymbolEv+0x3c)[0x7f865013f0]
/lib/aarch64-linux-gnu/libLLVM-14.so.1(_ZN4llvm16MCDwarfLineEntry4makeEPNS_10MCStreamerEPNS_9MCSectionE+0x3c)[0x7f86509ec0]
/lib/aarch64-linux-gnu/libLLVM-14.so.1(_ZN4llvm16MCObjectStreamer19emitInstructionImplERKNS_6MCInstERKNS_15MCSubtargetInfoE+0x94)[0x7f86526934]
/lib/aarch64-linux-gnu/libLLVM-14.so.1(_ZN4llvm16MCObjectStreamer15emitInstructionERKNS_6MCInstERKNS_15MCSubtargetInfoE+0x140)[0x7f86526858]
/lib/aarch64-linux-gnu/libLLVM-14.so.1(+0x28adf50)[0x7f86b7df50]
/lib/aarch64-linux-gnu/libLLVM-14.so.1(+0x2767f08)[0x7f86a37f08]
/lib/aarch64-linux-gnu/libLLVM-14.so.1(_ZN4llvm10AsmPrinter16emitFunctionBodyEv+0xa10)[0x7f8590ef30]
/lib/aarch64-linux-gnu/libLLVM-14.so.1(+0x2764ba4)[0x7f86a34ba4]
/lib/aarch64-linux-gnu/libLLVM-14.so.1(_ZN4llvm19MachineFunctionPass13runOnFunctionERNS_8FunctionE+0x140)[0x7f854c122c]
/lib/aarch64-linux-gnu/libLLVM-14.so.1(_ZN4llvm13FPPassManager13runOnFunctionERNS_8FunctionE+0x26c)[0x7f852a4390]
/lib/aarch64-linux-gnu/libLLVM-14.so.1(_ZN4llvm13FPPassManager11runOnModuleERNS_6ModuleE+0x3c)[0x7f852aaf70]
/lib/aarch64-linux-gnu/libLLVM-14.so.1(_ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE+0x7b4)[0x7f852a4d98]
/lib/aarch64-linux-gnu/librustc_driver-fe2d3f0c4114cdf4.so(+0x951d8c)[0x7f8b211d8c]
/lib/aarch64-linux-gnu/librustc_driver-fe2d3f0c4114cdf4.so(+0x92c114)[0x7f8b1ec114]
/lib/aarch64-linux-gnu/librustc_driver-fe2d3f0c4114cdf4.so(+0x930d88)[0x7f8b1f0d88]
/lib/aarch64-linux-gnu/librustc_driver-fe2d3f0c4114cdf4.so(+0x925db4)[0x7f8b1e5db4]
/lib/aarch64-linux-gnu/librustc_driver-fe2d3f0c4114cdf4.so(+0x92104c)[0x7f8b1e104c]
/lib/aarch64-linux-gnu/librustc_driver-fe2d3f0c4114cdf4.so(+0x80c0c0)[0x7f8b0cc0c0]
/lib/aarch64-linux-gnu/librustc_driver-fe2d3f0c4114cdf4.so(+0x8f6608)[0x7f8b1b6608]
/lib/aarch64-linux-gnu/libstd-4e83e61126e325db.so(rust_metadata_std_aeb45179e3b08762+0x9ffec)[0x7f8a7affec]
/lib/aarch64-linux-gnu/libc.so.6(+0x7ee58)[0x7f8a5dee58]
/lib/aarch64-linux-gnu/libc.so.6(+0xe7f9c)[0x7f8a647f9c]
compiling prism/serialize.c

make: *** [yjit/yjit.mk:26: yjit/target/release/libyjit.a] Segmentation fault
make: *** Waiting for unfinished jobs....
external command failed with status 2

BUILD FAILED (Debian 12 on aarch64 using ruby-build 20240612)

[maximecb]

Hi @arnlen. We might need your help to debug this. Are you able to figure out where it is crashing?

You should be able to run make V=1 to log the commands being executed, this would tell us which command is crashing.

I'm also wondering if it would work with a different version of rustc if the rust compiler is crashing.

[aidenfoxivey]

I gave it a go on my RPi5. (It's not urgent for me to have it running there- but figured I'd give it a shot.)

Didn't seem to cooperate with me compiling with the default version. I get this output when running rbenv install 3.3.3 --verbose:

/usr/bin/ld: yjit/target/release/libyjit.o: in function `__multc3':
/cargo/registry/src/index.crates.io-6f17d22bba15001f/compiler_builtins-0.1.108/./lib/builtins/multc3.c:33: undefined reference to `__builtin_copysignq'
/usr/bin/ld: /cargo/registry/src/index.crates.io-6f17d22bba15001f/compiler_builtins-0.1.108/./lib/builtins/multc3.c:34: undefined reference to `__builtin_copysignq'
/usr/bin/ld: /cargo/registry/src/index.crates.io-6f17d22bba15001f/compiler_builtins-0.1.108/./lib/builtins/multc3.c:42: undefined reference to `__builtin_copysignq'
/usr/bin/ld: /cargo/registry/src/index.crates.io-6f17d22bba15001f/compiler_builtins-0.1.108/./lib/builtins/multc3.c:43: undefined reference to `__builtin_copysignq'
/usr/bin/ld: /cargo/registry/src/index.crates.io-6f17d22bba15001f/compiler_builtins-0.1.108/./lib/builtins/multc3.c:47: undefined reference to `__builtin_copysignq'
/usr/bin/ld: yjit/target/release/libyjit.o:/cargo/registry/src/index.crates.io-6f17d22bba15001f/compiler_builtins-0.1.108/./lib/builtins/multc3.c:38: more undefined references to `__builtin_copysignq' follow

I assume this is a result of rustc's process since it's in the libyjit.o object. It seems that __multc3 is for complex long double multiplication. As far as I know, __builtin_copysignq is not something you'd even find on ARM64.

I'm not sure I quite know where the bug here would be, but I'll continue investigating.

(Please let me know if my input isn't helpful here - I don't want to be a pain.)

[maximecb]

(Please let me know if my input isn't helpful here - I don't want to be a pain.)

No worries, it would be good to get this problem fix, but we may not have the bandwidth to investigate it, so any help and input you can provide is welcome :)

I remember there being an issue with builtins that ended up being a problem with the rust toolchain @XrXr

@aidenfoxivey any chance you can upgrade your rustc to the latest?

[aidenfoxivey]

@maximecb My rustc seems to be rustc 1.78.0 (9b00956e5 2024-04-29). Should I try running 1.82 instead?

[aidenfoxivey]

Ah whoops, 1.82 is tomorrow! I'll try with 1.81

[aidenfoxivey]

Yup! Seems that running with 1.81 solves both installing with 3.3.0 and 3.3.3.

warning: this method call resolves to `<&Box<[T]> as IntoIterator>::into_iter` (due to backwards compatibility), but will resolve to `<Box<[T]> as IntoIterator>::into_iter` in Rust 2024
   --> ./yjit/src/core.rs:982:49
    |
982 |         formatter.debug_list().entries(branches.into_iter()).finish()
    |                                                 ^^^^^^^^^
    |
    = warning: this changes meaning in Rust 2024
    = note: `#[warn(boxed_slice_into_iter)]` on by default
hel
    |
982 |         formatter.debug_list().entries(branches.iter()).finish()
    |                                                 ~~~~
help: or use `IntoIterator::into_iter(..)` instead of `.into_iter()` to explicitly iterate by value
    |
982 |         formatter.debug_list().entries(IntoIterator::into_iter(branches)).finish()
    |                                        ++++++++++++++++++++++++        ~

I did notice this too. Is this something I could submit a micro-PR to change? I presume expected behaviour would be to use iter().

[XrXr]

I'm glad that it worked for you! OP should also try with the latest rustc. Looks like rustc is crashing during an LLVM call.

Is this something I could submit a micro-PR to change?

Thanks for thinking about contributing! But we've already fixed that particular warning.