Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When accessing the data pointer for an array, first decay it to a Derived Pointer #54335

Merged
merged 3 commits into from
May 3, 2024

Conversation

gbaraldi
Copy link
Member

@gbaraldi gbaraldi commented May 2, 2024

Fixes #54266, I've not yet minimized something to put into a test but in any case we can add the large test since it executes quite quickly.

This also enables IR verification with Strong=true when building with assertions, this would've caught this bug much earlier.

@gbaraldi gbaraldi requested a review from vtjnash May 2, 2024 20:11
@Keno
Copy link
Member

Keno commented May 2, 2024

A pointer GEP in addrspace 10 is basically never legal (because each object is a separate allocation and there's only one addrspace 10 pointer that's valid in that allocation). Can we add this to the verifier? Then adding a test for this will also become easy.

@Keno
Copy link
Member

Keno commented May 2, 2024

Actually, we already have it

void GCInvariantVerifier::visitGetElementPtrInst(GetElementPtrInst &GEP) {
:

    /* We're actually ok with GEPs here, as long as they don't feed into any
       uses. Upstream is currently still debating whether CAST(GEP) == GEP(CAST).
       In the frontend, we always perform CAST(GEP), so while we can enforce
       this invariant when we run directly after the frontend (Strong == 1),
       the optimizer will introduce the other form. Thus, we need to allow it
       while upstream hasn't decided whether the optimizer is allowed to
       introduce these.
       */
    if (Strong) {
        Check(AS != AddressSpace::Tracked,
             "GC tracked values may not appear in GEP expressions."
             " You may have to decay the value first", &GEP);
    }

But clearly it's not triggering in this example, so we may need to enhance it.

@gbaraldi
Copy link
Member Author

gbaraldi commented May 2, 2024

@Keno the issue is that the check is behind the Strong flag, you have a comment there as to why we may disable this, do you know if we can just remove that check?
Making the check unconditional does find this.

@Keno
Copy link
Member

Keno commented May 2, 2024

As the comment indicates, the Strong version of this should run on the IR that we output from the frontend, which is supposed to catch this issue.

@Keno
Copy link
Member

Keno commented May 2, 2024

I don't believe LLVM has decided the pointercast vs gep issue.

@gbaraldi
Copy link
Member Author

gbaraldi commented May 2, 2024

So we should run the verifier twice then? Once before optimization and once after with Strong off?

@Keno
Copy link
Member

Keno commented May 2, 2024

Yes, that's how it's designed (in debug mode only of course). It's possible we lost one of the passes.

@gbaraldi
Copy link
Member Author

gbaraldi commented May 2, 2024

I'm tempted to move this check to run when assertions are on instead of just debug. Debug is not part of CI nor PkgEval and that pass isn't particularly expensive, it should be O(N) with a small constant

@Keno
Copy link
Member

Keno commented May 2, 2024

That's fine.

@gbaraldi gbaraldi merged commit 9d59ecc into master May 3, 2024
7 checks passed
@gbaraldi gbaraldi deleted the gb/addrspace_fun branch May 3, 2024 17:04
@gbaraldi gbaraldi added the backport 1.11 Change should be backported to release-1.11 label May 3, 2024
@KristofferC KristofferC mentioned this pull request May 6, 2024
59 tasks
KristofferC pushed a commit that referenced this pull request May 6, 2024
…ived Pointer (#54335)

Fixes #54266, I've not yet
minimized something to put into a test but in any case we can add the
large test since it executes quite quickly.

This also enables IR verification with `Strong=true` when building with
assertions, this would've caught this bug much earlier.

(cherry picked from commit 9d59ecc)
KristofferC added a commit that referenced this pull request May 28, 2024
Backported PRs:
- [x] #53665 <!-- use afoldl instead of tail recursion for tuples -->
- [x] #53976 <!-- LinearAlgebra: LazyString in interpolated error
messages -->
- [x] #54005 <!-- make `view(::Memory, ::Colon)` produce a Vector -->
- [x] #54010 <!-- Overload `Base.literal_pow` for `AbstractQ` -->
- [x] #54069 <!-- Allow PrecompileTools to see MI's inferred by foreign
abstract interpreters -->
- [x] #53750 <!-- inference correctness: fields and globals can revert
to undef -->
- [x] #53984 <!-- Profile: fix heap snapshot is valid char check -->
- [x] #54102 <!-- Explicitly compute stride in unaliascopy for SubArray
-->
- [x] #54070 <!-- Fix integer overflow in `skip(s::IOBuffer,
typemax(Int64))` -->
- [x] #54013 <!-- Support case-changes to Annotated{String,Char}s -->
- [x] #53941 <!-- Fix writing of AnnotatedChars to AnnotatedIOBuffer -->
- [x] #54137 <!-- Fix typo in docs for `partialsortperm` -->
- [x] #54129 <!-- use correct size when creating output data from an
IOBuffer -->
- [x] #54153 <!-- Fixup IdSet docstring -->
- [x] #54143 <!-- Fix `make install` from tarballs -->
- [x] #54151 <!-- LinearAlgebra: Correct zero element in
`_generic_matvecmul!` for block adj/trans -->
- [x] #54213 <!-- Add `public` statement to `Base.GC` -->
- [x] #54222 <!-- Utilize correct tbaa when emitting stores of unions.
-->
- [x] #54233 <!-- set MAX_OS_WRITE on unix -->
- [x] #54255 <!-- fix `_checked_mul_dims` in the presence of 0s and
overflow. -->
- [x] #54259 <!-- Fix typo in `readuntil` -->
- [x] #54251 <!-- fix typo in gc_mark_memory8 when chunking a large
array -->
- [x] #54276 <!-- Fix solve for complex `Hermitian` with non-vanishing
imaginary part on diagonal -->
- [x] #54248 <!-- ensure package callbacks are invoked when no valid
precompile file exists for an "auto loaded" stdlib -->
- [x] #54308 <!-- Implement eval-able AnnotatedString 2-arg show -->
- [x] #54302 <!-- Specialised substring equality for annotated strs -->
- [x] #54243 <!-- prevent `package_callbacks` to run multiple time for a
single package -->
- [x] #54350 <!-- add a precompile signature to Artifacts code that is
used by JLLs -->
- [x] #54331 <!-- correctly track freed bytes in
jl_genericmemory_to_string -->
- [x] #53509 <!-- revert moving "creating packages" from Pkg.jl -->
- [x] #54335 <!-- When accessing the data pointer for an array, first
decay it to a Derived Pointer -->
- [x] #54239 <!-- Make sure `fieldcount` constant-folds for `Tuple{...}`
-->
- [x] #54288
- [x] #54067
- [x] #53715 <!-- Add read/write specialisation for IOContext{AnnIO} -->
- [x] #54289 <!-- Rework annotation ordering/optimisations -->
- [x] #53815 <!-- create phantom task for GC threads -->
- [x] #54130 <!-- inference: handle `LimitedAccuracy` in
`handle_global_assignment!` -->
- [x] #54428 <!-- Move ConsoleLogging.jl into Base -->
- [x] #54332 <!-- Revert "add unsetindex support to more copyto methods
(#51760)" -->
- [x] #53826 <!-- Make all command-line options documented in all
related files -->
- [x] #54465 <!-- typeintersect: conservative typevar subtitution during
`finish_unionall` -->
- [x] #54514 <!-- typeintersect: followup cleanup for the nothrow path
of type instantiation -->
- [x] #54499 <!-- make `@doc x` work without REPL loaded -->
- [x] #54210 <!-- attach finalizer in `mmap` to the correct object -->
- [x] #54359 <!-- Pkg REPL: cache `pkg_mode` lookup -->

Non-merged PRs with backport label:
- [ ] #54471 <!-- Actually setup jit targets when compiling
packageimages instead of targeting only one -->
- [ ] #54457 <!-- Make `String(::Memory)` copy -->
- [ ] #54323 <!-- inference: fix too conservative effects for recursive
cycles -->
- [ ] #54322 <!-- effects: add new `@consistent_overlay` macro -->
- [ ] #54191 <!-- make `AbstractPipe` public -->
- [ ] #53957 <!-- tweak how filtering is done for what packages should
be precompiled -->
- [ ] #53882 <!-- Warn about cycles in extension precompilation -->
- [ ] #53707 <!-- Make ScopedValue public -->
- [ ] #53452 <!-- RFC: allow Tuple{Union{}}, returning Union{} -->
- [ ] #53402 <!-- Add `jl_getaffinity` and `jl_setaffinity` -->
- [ ] #53286 <!-- Raise an error when using `include_dependency` with
non-existent file or directory -->
- [ ] #52694 <!-- Reinstate similar for AbstractQ for backward
compatibility -->
- [ ] #51479 <!-- prevent code loading from lookin in the versioned
environment when building Julia -->
@KristofferC KristofferC removed the backport 1.11 Change should be backported to release-1.11 label May 28, 2024
lazarusA pushed a commit to lazarusA/julia that referenced this pull request Jul 12, 2024
…ived Pointer (JuliaLang#54335)

Fixes JuliaLang#54266, I've not yet
minimized something to put into a test but in any case we can add the
large test since it executes quite quickly.

This also enables IR verification with `Strong=true` when building with
assertions, this would've caught this bug much earlier.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

segfault in jl_datatype_layout while constructing Memory
4 participants