Faster incremental sysimg rebuilds

[Keno]

Faster incremental sysimg rebuilds

Recent improvements in precompilation have improved compile
time issues like ttfp quite significantly. However, it is
still significantly faster to just build a system image,
in which case ttfp is basically instant. The difference is
primarily due to us not being able to store native code in
.ji files as well as invalidations of previously loaded code
requiring recompilation. In the long term these issues can
be overcome, but in the short term, I think we should try
to leverage system images more heavily, since they already
basically solve the problem. I believe the reason people
aren't really using system images is three-fold

1. System images take a long time to compile
2. The system image workflow is pretty manual
3. System images don't play well with pkg updates etc.

Thus, my evil plan to improve the situation is

1. Make system images build faster
2. Add an autoload annotation to Project.toml files.
   If present, julia will hash the manifest and look
   for any matching system image in ~/.julia/sysimages.
3. Make system images build even faster

The idea is that for the standard workflow where people
just use plain julia with the default environment or
julia --project system images would be just loaded
automatically, thus reducing the barrier to entry.

In the initial version, there is no automatic rebuild
of these system images - they would still be built manually
with PkgCompiler, but at least the loading side would
be automatic and hopefully the build will be fast enough
that people will actually be willing to wait.
Eventually the rebuild could also be automatic
(maybe even in the background).

The major drawback of this plan is that system images will
start with all packages already loaded (even if their
bindings aren't present in Main). This will require some
workflow adjustments. I think it'll probably turn out fine,
but it's worth highlighting.

This PR is step 1 in this direction. It provides the ability
to rebuild system images much faster. The key observation
is that most of the time in sysimage build is spent in LLVM
generating native code (serializing julia's data structures
is quite fast). Thus if we can re-use the code already
generated for the system image we're currently running, we'll
save a fair amount of time.

Unfortunately, this is not 100% straightforward since we were
assuming that no linking happens in a number of places. This
PR hacks around that, but it is not a particularly satisfying
long term solution. That said, it should work fine, and I think
it's worth doing, so that we can explore the workflow
adjustments that would rely on this.

With that said, here's how to use this (at the low level, of
course PkgCompiler would just handle this)

$ mkdir chained
$ time ./usr/bin/julia --sysimage-native-code=chained --sysimage=usr/lib/julia/sys.so --output-o chained/chained.o.a -e 'Base.__init_build();'
real	0m9.633s
user	0m8.613s
sys	0m1.020s
$ cd chained
$ cp ../usr/lib/julia/sys-o.a . # Get the -o.a from the old sysimage
$ ar x sys-o.a # Extract it into text.o and data.o
$ rm data.o # rm the serialized sysimg data
$ mv text.o text-old.o
$ llvm-objcopy --remove-section .data.jl.sysimg_link text-old.o # rm the link between the native code and the old sysimg data
$ ar x chained.o.a # Extract new sysimage files
$ gcc -shared -o chained.so text.o data.o text-old.o # Link everything
$ ../julia --sysimage=chained.so

As can be seen, regenerating the system image took about 9s (the
subsequent commands aren't timed here, but take less than a second total).
This compares very favorably with a non-chained sysimg rebuild:

time ./usr/bin/julia --sysimage=usr/lib/julia/sys.so --output-o nonchained.o.a -e 'Base.__init_build();'

real	2m42.667s
user	2m39.211s
sys	0m3.452s

Of course if you do load additional packages, the extra code
does still need to be compiled, so e.g. building a system image
for Plots goes from 3 mins to 1 mins (building all of plots,
plus everything in base that got invalidated). That is still all in
LLVM though - it should be relatively straightforward to
multithread that after this PR (since linking the sysimg
in multiple pieces is allowed). That part is not implemented
yet though.

[NHDaly]

Amazing! Thank you for tackling this, @Keno! It sounds very exciting!

In case you haven't seen this already, regarding speeding up PackageCompiler, I've asked in the past about why PackageCompiler currently ends up compiling everything to native code twice, where we round-trip through a text file to record the compilations: JuliaLang/PackageCompiler.jl#486. Just wanted to float this past your vision in case you hadn't seen it. Kristoffer provided good explanation, but it seems like something that could be improved with a bit of design work.

[Keno]

This can basically address that as well. You can build the whole sysimg without precompiles, then dump out your precompiles with that and then use this mechanism to build a chained sysimg as before much faster.

[Keno]

@timholy one application that comes to mind is speeding up development of Base itself. Could we have a mode of Revise where it serializes what needs to be Revised in some easy to load format, and then use this to quickly update an existing system image (without Revise itself showing up in the system image)?

[timholy]

Just noticed #40414 (comment). Sure, that would be pretty easy to do in principle. What exactly would it look like?

File an issue at Revise when you want this; my impression is that we're not yet at a place where this will make a difference.

[timholy]

I have to say, this triggers my love/hate relationship with https://github.com/JuliaLang/PackageCompiler.jl. I totally get why it's necessary to have it, but at the same time its existence is probably what's let us get away for so long without just implementing native-code caching in package .ji files. I'd rather just fix that. Are we really so far from that goal? It just doesn't seem like it should be all that insurmountable. I'm on a bit of a close-the-precompile-issues rampage right now. There really aren't that many issues per se, but we'll still need some things (the method.roots problem #32705, and the issue of whether to store non-internal CodeInstances) that are pretty big.

[jpsamaroo]

Alternatively, could we put the native code into shared libraries, and load them when we load .ji files? It could improve the situation of calling Julia from C, since we'd have an obvious place to emit ccall-able methods, and they could potentially remain available without the runtime being started (if they don't depend on the runtime).

[timholy]

I'm not really sure of the right implementation, mostly because I've never actually looked at the format of a shared library file. But that seems pretty sensible. Once we can cache external MethodInstances & CodeInstances (in our current no-native-code format), AFAICT the main remaining job is doing the work of the linker. If we can rely on external tools, that seems likely to be a win.

[ViralBShah]

@Keno is this the PR you said can be rebased and brought back?