Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid side effects when precompiling find_artifact_dir #66

Merged
merged 1 commit into from
Aug 30, 2024

Conversation

giordano
Copy link
Member

With an explicit call to find_artifact_dir() we were actually fetching lazy artifacts during precompilation, which is typically undesired, besides hitting a bug in Downloads.jl in certain versions of Julia (up to 1.10.4) which causes the download of the artifact to hang. Instead, with Base.precompile we retain the caching of native code, without causing extra downloads during precompilation.

Some benchmarks, same as in #53. Before this PR:

julia> @time_imports using LLVMExtra_jll
      0.2 ms  LazyArtifacts
     13.6 ms  Preferences
      0.4 ms  JLLWrappers
               ┌ 6.8 ms LLVMExtra_jll.__init__() 59.45% compilation time
     50.7 ms  LLVMExtra_jll 92.31% compilation time

With this PR:

julia> @time_imports using LLVMExtra_jll
      0.2 ms  LazyArtifacts
     14.2 ms  Preferences
      0.5 ms  JLLWrappers
               ┌ 6.7 ms LLVMExtra_jll.__init__() 59.63% compilation time
     50.4 ms  LLVMExtra_jll 92.59% compilation time

So there's no penalty in using Base.precompile. If I remove the forced precompilation entirely:

julia> @time_imports using LLVMExtra_jll
      0.2 ms  LazyArtifacts
     10.2 ms  Preferences
      0.4 ms  JLLWrappers
               ┌ 4.5 ms LLVMExtra_jll.__init__() 58.45% compilation time
     52.4 ms  LLVMExtra_jll 94.76% compilation time

This is slower than with the precompilation call, but not as bad as in #53.

Also, with this PR:

(@v1.10) pkg> add IntelOpenMP_jll
   Resolving package versions...
   Installed IntelOpenMP_jll ─ v2024.2.1+0
    Updating `~/.julia/environments/v1.10/Project.toml`
  [1d5cc7b8] + IntelOpenMP_jll v2024.2.1+0
    Updating `~/.julia/environments/v1.10/Manifest.toml`
  [1d5cc7b8] + IntelOpenMP_jll v2024.2.1+0
Precompiling project...
  1 dependency successfully precompiled in 1 seconds. 56 already precompiled.

julia> using IntelOpenMP_jll
  Downloaded artifact: IntelOpenMP

julia>

Download happens at using-time, not during precompilation, as intended. Note that I'm using Julia 1.10.4, which would also cause the precompilation to last at least 30 seconds because of the Downloads.jl bug.

Fix #65.

With an explicit call to `find_artifact_dir()` we were actually fetching lazy
artifacts during precompilation, which is typically undesired, besides hitting a
bug in Downloads.jl in certain versions of Julia (up to 1.10.4) which causes the
download of the artifact to hang.  Instead, with `Base.precompile` we retain the
caching of native code, without causing extra downloads during precompilation.
Copy link
Contributor

@topolarity topolarity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @giordano !

@staticfloat staticfloat merged commit c121344 into master Aug 30, 2024
35 of 49 checks passed
@staticfloat staticfloat deleted the mg/precompile-find-artifact-dir branch August 30, 2024 16:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Lazy artifacts cause precompilation-time warning
4 participants