move out Profile from the sysimage #49132

KristofferC · 2023-03-24T11:24:04Z

Loads fast anyway (edit: a bit slower when adding the precompile stuff):

julia> @time using Profile
  0.008181 seconds (13.36 k allocations: 900.847 KiB)

IanButterworth · 2023-03-24T12:12:55Z

Profile has stuff precompiled in the repl precompiler

KristofferC · 2023-03-24T12:18:08Z

Why does

julia/contrib/generate_precompile.jl

Line 240 in db7971f

 repl_script = Profile.precompile_script * repl_script # do larger workloads first for better parallelization 

have to run in the REPL?

IanButterworth · 2023-03-24T12:21:00Z

I guess it doesn't.

KristofferC · 2023-03-24T13:05:49Z

I moved the precompile stuff from generate_precompile to Profile.

KristofferC · 2023-03-24T13:29:58Z

Need to figure out how this will work with the hotkeys for profiling a running application.

IanButterworth · 2023-03-24T16:29:42Z

Need to figure out how this will work with the hotkeys for profiling a running application.

Yeah. It would be great to figure out how to maintain the lack of requirement to explicitly load Profile for the profiling of running code to work.

Options I can see:

Move the Profile.Print functions to c code so it can be called in signals-*.c - It's a lot of code though
Figure out how to load Profile from signals-*.c if it isn't loaded already when a profile is requested
Just require that Profile is loaded first, so people who want the feature need to remember to put it into their startup.jl. This would be a shame IMO.
Just load Profile on startup, even if it's not in the sysimage
On this PR it's pretty quick!

julia> @time using Profile
  0.077875 seconds (146.53 k allocations: 11.794 MiB)

KristofferC · 2023-03-24T17:06:56Z

Just load Profile on startup, even if it's not in the sysimage. On this PR it's pretty quick!

Well, it is slower than just starting julia itself so I don't think that is an option.

IanButterworth · 2023-03-24T18:34:10Z

Ok, I found and pushed an alternative fix which is to move the listener into Base and load Profile there when needed

IanButterworth · 2023-03-24T19:00:06Z

I'm still hitting precompilation

julia> using Profile
[ Info: Precompiling Profile [9abbd945-dff8-562f-b5e8-e1ebf5ef1b79]

KristofferC · 2023-03-24T20:17:57Z

I'm still hitting precompilation

It seemed to work for me.

julia> ENV["JULIA_DEBUG"] = "loading";

julia> using Profile
┌ Debug: Rejecting cache file /Users/kristoffercarlsson/.julia/compiled/v1.10/Profile/nGhxz_pvFUp.ji for Profile [9abbd945-dff8-562f-b5e8-e1ebf5ef1b79] since the flags are mismatched
│   current session: use_pkgimages = true, debug_level = 1, check_bounds = 0, inline = true, opt_level = 2
│   cache file:      use_pkgimages = true, debug_level = 1, check_bounds = 1, inline = true, opt_level = 2
└ @ Base loading.jl:2724
┌ Debug: Loading object cache file /Users/kristoffercarlsson/.julia/compiled/v1.10/Profile/nGhxz_ggTXk.dylib for Profile [9abbd945-dff8-562f-b5e8-e1ebf5ef1b79]
└ @ Base loading.jl:1010

These files were created by the pkgimage builder make step.

IanButterworth · 2023-03-25T04:37:06Z

I didn't cleanall before build. It could be a make issue

IanButterworth · 2023-03-25T04:39:08Z

Wait, but neither of those cache files are in your julia dir, they're in your depot. Shouldn't they be bundled with Julia?

KristofferC · 2023-03-25T06:11:15Z

I think only when you are making a release they get bundled by setting the depot. But not sure..

IanButterworth · 2023-03-26T13:51:41Z

After a cleanall build

julia> using Profile
┌ Debug: Rejecting cache file /Users/ian/.julia/compiled/v1.10/Profile/nGhxz_wG2DL.ji because module Core [top-level] is already loaded and incompatible.
└ @ Base loading.jl:2779
[ Info: Precompiling Profile [9abbd945-dff8-562f-b5e8-e1ebf5ef1b79]
┌ Debug: Loading object cache file /Users/ian/.julia/compiled/v1.10/Profile/nGhxz_wG2DL.dylib for Profile [9abbd945-dff8-562f-b5e8-e1ebf5ef1b79]
└ @ Base loading.jl:1010

Also, all of the stdlib/*.release.image files are zero bytes. Is that expected?

This is intel MacOS if that's relevant

cc. @vchuravy

vchuravy · 2023-03-26T17:15:30Z

Also, all of the stdlib/*.release.image files are zero bytes. Is that expected?

These are marker files. It's a make technique when you don't know the output file of a rule.

I set JULIA_DEPOT_PATH := $build_prefix/share/julia. Can you check if it contains the files you expected? That should be the same as Sys.STDLIB.

IanButterworth · 2023-03-26T22:22:15Z

Great. That fixed it

julia> using Profile
┌ Debug: Rejecting cache file /Users/ian/Documents/GitHub/julia/usr/share/julia/compiled/v1.10/Profile/nGhxz_rikHs.ji for Profile [9abbd945-dff8-562f-b5e8-e1ebf5ef1b79] since the flags are mismatched
│   current session: use_pkgimages = true, debug_level = 1, check_bounds = 0, inline = true, opt_level = 2
│   cache file:      use_pkgimages = true, debug_level = 1, check_bounds = 1, inline = true, opt_level = 2
└ @ Base loading.jl:2724
┌ Debug: Loading object cache file /Users/ian/Documents/GitHub/julia/usr/share/julia/compiled/v1.10/Profile/nGhxz_KZhni.dylib for Profile [9abbd945-dff8-562f-b5e8-e1ebf5ef1b79]
└ @ Base loading.jl:1010

Very minor but perhaps the check_bounds = 0 image should be built last so it has the latest mtime and is tried first?

IanButterworth · 2023-03-27T11:56:29Z

Something's not quite right, perhaps with the make setup?

After the last commit and a make cleanall then make I get

julia> using Profile
┌ Debug: Rejecting cache file /Users/ian/.julia/compiled/v1.10/Profile/nGhxz_wG2DL.ji because module Core [top-level] is already loaded and incompatible.
└ @ Base loading.jl:2779
[ Info: Precompiling Profile [9abbd945-dff8-562f-b5e8-e1ebf5ef1b79]
┌ Debug: Loading object cache file /Users/ian/.julia/compiled/v1.10/Profile/nGhxz_wG2DL.dylib for Profile [9abbd945-dff8-562f-b5e8-e1ebf5ef1b79]
└ @ Base loading.jl:1010

So it's not even trying to find cachefiles in /Users/ian/Documents/GitHub/julia/usr/share/julia/compiled

vchuravy · 2023-03-27T12:07:18Z

That seems right? We start at the lowest depot and then go up from there? What happens if you remove compiled/Profile?

IanButterworth · 2023-03-27T12:39:40Z

Oh.. so if it finds something it can't use in a depot, it doesn't search in deeper depots?

IanButterworth · 2023-03-27T12:41:42Z

That didn't work

julia> using Profile
[ Info: Precompiling Profile [9abbd945-dff8-562f-b5e8-e1ebf5ef1b79]
┌ Debug: Loading object cache file /Users/ian/.julia/compiled/v1.10/Profile/nGhxz_wG2DL.dylib for Profile [9abbd945-dff8-562f-b5e8-e1ebf5ef1b79]
└ @ Base loading.jl:1010

julia> Base.DEPOT_PATH
3-element Vector{String}:
 "/Users/ian/.julia"
 "/Users/ian/Documents/GitHub/julia/usr/local/share/julia"
 "/Users/ian/Documents/GitHub/julia/usr/share/julia"

% ls /Users/ian/Documents/GitHub/julia/usr/share/julia/compiled/v1.10
DelimitedFiles	SparseArrays

IanButterworth · 2023-03-27T18:00:32Z

Fixing that missing comma didn't fix it

% make cleanall
...
% make -j6
...
Outputting sysimage file...
Output ──────  60.238698 seconds
    LINK usr/lib/julia/sys.dylib
    JULIA stdlib/SparseArrays.release.image
    JULIA stdlib/DelimitedFiles.release.image
    JULIA stdlib/DelimitedFiles.release.image
    JULIA stdlib/SparseArrays.release.image

% ls /Users/ian/Documents/GitHub/julia/usr/share/julia/compiled/v1.10
DelimitedFiles	SparseArrays

vchuravy · 2023-03-27T19:13:33Z

Looking at the artifiact from one of the build jobs:

IanButterworth · 2023-03-27T20:06:17Z

Works well for me now. The ctrl-t profile prints quickly

vchuravy · 2023-03-28T15:23:33Z

Good to go? Or should I move out the Bugfixes into a separate PR?

IanButterworth · 2023-03-30T14:32:17Z

Or codesigning. It might be that the error happens on the same machine, it's just that locally I only make

IanButterworth · 2023-03-30T17:07:09Z

Btw, the same checksum failure also happens for the other non-sysimage stdlibs like SparseArrays and they end up being re-precompiled

IanButterworth · 2023-03-31T01:56:39Z

Ok, so as expected ignoring the checksum failure for pkgimage stdlibs on macos works, so the files are good, just changed somehow.

How can we debug this further?

IanButterworth · 2023-03-31T15:38:39Z

The mtimes of the dylibs correspond to codesigning, so that must be it.

ian@IanMBP julia-f65df74f4d % ls -lT share/julia/compiled/v1.10/Profile                           
total 14776
-rwxr-xr-x@ 1 ian  staff  3728224 Mar 29 10:22:10 2023 nGhxz_pQJuF.dylib
drwxr-xr-x@ 3 ian  staff       96 Mar 29 10:22:06 2023 nGhxz_pQJuF.dylib.dSYM
-rw-r--r--@ 1 ian  staff    58898 Mar 29 10:22:06 2023 nGhxz_pQJuF.ji
-rwxr-xr-x@ 1 ian  staff  3708096 Mar 29 10:22:10 2023 nGhxz_x4TlO.dylib
drwxr-xr-x@ 3 ian  staff       96 Mar 29 10:22:06 2023 nGhxz_x4TlO.dylib.dSYM
-rw-r--r--@ 1 ian  staff    58898 Mar 29 10:22:06 2023 nGhxz_x4TlO.ji

  | 2023-03-29 10:22:10 EDT | Codesigning /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-x64-4.0/build/default-macmini-x64-4-0/julialang/julia-master/julia-f65df74f4d/share/julia/compiled/v1.10/Profile/nGhxz_pQJuF.dylib...
  | 2023-03-29 10:22:10 EDT | /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-x64-4.0/build/default-macmini-x64-4-0/julialang/julia-master/julia-f65df74f4d/share/julia/compiled/v1.10/Profile/nGhxz_pQJuF.dylib: signed Mach-O thin (x86_64) [nGhxz_pQJuF-555549444c4c44cc55553144a1181b3ee2f4af2a]
  | 2023-03-29 10:22:10 EDT | Codesigning /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-x64-4.0/build/default-macmini-x64-4-0/julialang/julia-master/julia-f65df74f4d/share/julia/compiled/v1.10/Profile/nGhxz_x4TlO.dylib...
  | 2023-03-29 10:22:10 EDT | /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-x64-4.0/build/default-macmini-x64-4-0/julialang/julia-master/julia-f65df74f4d/share/julia/compiled/v1.10/Profile/nGhxz_x4TlO.dylib: signed Mach-O thin (x86_64) [nGhxz_x4TlO-555549444c4c444a55553144a1e4a5d673f79711]

@staticfloat is it known that codesigning modifies dylibs? If so do we need to just overwrite checksums at the end of the .ji files after codesigning?

staticfloat · 2023-03-31T17:44:20Z

@staticfloat is it known that codesigning modifies dylibs?

Yes, that's the whole point of codesigning; to modify the dylibs by adding a cryptographic signature to them. I thought the .dylib files would have a hash of the .ji files, not the other way around. Aren't the .dylib files created after the .ji files?

vchuravy · 2023-03-31T18:31:42Z

The ji contains very little information that the dylibs are dependent on, but we use the ji for quick rejection of cachefiles so we want to be able to detect corruption before we go to load the dylib (which we often can't unload)

IanButterworth · 2023-03-31T18:33:14Z

Yes, that's the whole point of codesigning

I had thought codesigning saved its information elsewhere

staticfloat · 2023-03-31T18:36:23Z

I see. Then yes, we need a convenient way to update the checksums inside of the .ji files.

IanButterworth · 2023-04-03T01:57:06Z

Now that #48545 is merged, this should just work.

Codesigning will still happen for .dmg distribution but that happens after tests and will be handled here JuliaCI/julia-buildkite#292

IanButterworth · 2023-04-03T19:27:22Z

Now that #48545 is merged, this should just work.

That was wrong, it needed JuliaCI/julia-buildkite#292 which is now merged, so I updated here

IanButterworth

Should be good to go now

giordano added the stdlib Julia's standard library label Mar 24, 2023

KristofferC marked this pull request as draft March 24, 2023 13:45

KristofferC force-pushed the kc/profile_sysimage branch from fab49f4 to 1847fba Compare March 24, 2023 16:08

KristofferC marked this pull request as ready for review March 24, 2023 20:40

vchuravy mentioned this pull request Mar 26, 2023

Export JULIA_DEPOT_PATH in pkgimage.mk #49161

Merged

IanButterworth approved these changes Mar 27, 2023

View reviewed changes

This was referenced Mar 31, 2023

Add contrib tool to update shipped pkgimage checksums after codesigning #49211

Closed

update macos stdlib pkgimage checksums after codesigning JuliaCI/julia-buildkite#292

Merged

IanButterworth force-pushed the kc/profile_sysimage branch 7 times, most recently from 439719c to e104895 Compare April 3, 2023 01:54

move out Profile from the sysimage

ca6fe29

IanButterworth force-pushed the kc/profile_sysimage branch from e104895 to ca6fe29 Compare April 3, 2023 19:26

IanButterworth approved these changes Apr 4, 2023

View reviewed changes

staticfloat approved these changes Apr 4, 2023

View reviewed changes

KristofferC merged commit b258051 into master Apr 4, 2023

KristofferC deleted the kc/profile_sysimage branch April 4, 2023 04:42

Xnartharax pushed a commit to Xnartharax/julia that referenced this pull request Apr 19, 2023

move out Profile from the sysimage (JuliaLang#49132)

259b348

mkitti mentioned this pull request Jun 1, 2023

Adding osx-arm support v3 conda-forge/julia-feedstock#252

Open

MilesCranmer added a commit to MilesCranmer/julia-feedstock that referenced this pull request Jun 1, 2023

Backport JuliaLang/julia#49132

97d3aad

MilesCranmer added a commit to MilesCranmer/julia-feedstock that referenced this pull request Jun 1, 2023

Manually backport JuliaLang/julia#49132

dce4669

KristofferC mentioned this pull request Aug 28, 2023

22% faster startup because sysimage is 33% smaller #51082

Closed

move out Profile from the sysimage #49132

move out Profile from the sysimage #49132

Conversation

KristofferC commented Mar 24, 2023 • edited Loading

IanButterworth commented Mar 24, 2023

KristofferC commented Mar 24, 2023

IanButterworth commented Mar 24, 2023

KristofferC commented Mar 24, 2023 • edited Loading

KristofferC commented Mar 24, 2023

IanButterworth commented Mar 24, 2023

KristofferC commented Mar 24, 2023

IanButterworth commented Mar 24, 2023

IanButterworth commented Mar 24, 2023

KristofferC commented Mar 24, 2023

IanButterworth commented Mar 25, 2023

IanButterworth commented Mar 25, 2023 • edited Loading

KristofferC commented Mar 25, 2023

IanButterworth commented Mar 26, 2023

vchuravy commented Mar 26, 2023

IanButterworth commented Mar 26, 2023

IanButterworth commented Mar 27, 2023

vchuravy commented Mar 27, 2023

IanButterworth commented Mar 27, 2023 • edited Loading

IanButterworth commented Mar 27, 2023 • edited Loading

IanButterworth commented Mar 27, 2023

vchuravy commented Mar 27, 2023

IanButterworth commented Mar 27, 2023

vchuravy commented Mar 28, 2023

IanButterworth commented Mar 30, 2023

IanButterworth commented Mar 30, 2023

IanButterworth commented Mar 31, 2023

IanButterworth commented Mar 31, 2023

staticfloat commented Mar 31, 2023

vchuravy commented Mar 31, 2023

IanButterworth commented Mar 31, 2023

staticfloat commented Mar 31, 2023

IanButterworth commented Apr 3, 2023

IanButterworth commented Apr 3, 2023

IanButterworth left a comment

Choose a reason for hiding this comment

KristofferC commented Mar 24, 2023 •

edited

Loading

KristofferC commented Mar 24, 2023 •

edited

Loading

IanButterworth commented Mar 25, 2023 •

edited

Loading

IanButterworth commented Mar 27, 2023 •

edited

Loading

IanButterworth commented Mar 27, 2023 •

edited

Loading