Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support LLVM 10 in GC checker #37153

Merged
merged 1 commit into from
Aug 24, 2020
Merged

Support LLVM 10 in GC checker #37153

merged 1 commit into from
Aug 24, 2020

Conversation

yuyichao
Copy link
Contributor

@yuyichao yuyichao commented Aug 22, 2020

Ref llvm/llvm-project@2f169e7
Ref llvm/llvm-project@6b85f8e

Also handle noreturn function call in GC frame counting,
and remove some old code for LLVM <= 7

@yuyichao yuyichao requested a review from Keno August 22, 2020 04:15
@vchuravy vchuravy mentioned this pull request Aug 22, 2020
3 tasks
@vchuravy
Copy link
Sponsor Member

Did a run with LLVM 10 and the GCChecker https://build.julialang.org/#/builders/65/builds/2239/steps/4/logs/stdio

@yuyichao
Copy link
Contributor Author

yuyichao commented Aug 23, 2020

The only failure I got locally is due to LLVM not able to infer that CallInst::getCalledOperand() returns non-NULL value and can be fixed with,

diff --git a/src/llvm-alloc-opt.cpp b/src/llvm-alloc-opt.cpp
index 6f2a69dea9..eb1abe932d 100644
--- a/src/llvm-alloc-opt.cpp
+++ b/src/llvm-alloc-opt.cpp
@@ -1318,6 +1318,7 @@ void Optimizer::splitOnStack(CallInst *orig_inst)
         }
         else if (auto call = dyn_cast<CallInst>(user)) {
             auto callee = call->getCalledOperand();
+            assert(callee);
             if (auto intrinsic = dyn_cast<IntrinsicInst>(call)) {
                 if (Intrinsic::ID id = intrinsic->getIntrinsicID()) {
                     if (id == Intrinsic::memset) {

@vchuravy
Copy link
Sponsor Member

vchuravy commented Aug 23, 2020

Hm, I am getting the same failure locally.

    ANALYZE /home/vchuravy/builds/julia-llvm10/src/clang-sa-jitlayers
In file included from /home/vchuravy/src/julia/src/jitlayers.cpp:8:
In file included from /home/vchuravy/builds/julia-llvm10/usr/include/llvm/Transforms/Utils/Cloning.h:22:
In file included from /home/vchuravy/builds/julia-llvm10/usr/include/llvm/Analysis/AliasAnalysis.h:45:
In file included from /home/vchuravy/builds/julia-llvm10/usr/include/llvm/Analysis/TargetLibraryInfo.h:19:
In file included from /home/vchuravy/builds/julia-llvm10/usr/include/llvm/IR/PassManager.h:46:
In file included from /home/vchuravy/builds/julia-llvm10/usr/include/llvm/IR/PassInstrumentation.h:57:
/home/vchuravy/builds/julia-llvm10/usr/include/llvm/ADT/FunctionExtras.h:142:5: error: Undefined or garbage value returned to caller
    return StorageUnion.OutOfLineStorage.StoragePtr;
    ^
/home/vchuravy/src/julia/src/jitlayers.cpp:482:5: note: Calling 'DebugObjectRegistrar::registerObject'
    registerObject(H, Object,
    ^~~~~~~~~~~~~~~~~~~~~~~~~
/home/vchuravy/src/julia/src/jitlayers.cpp:462:13: note: Assuming the condition is false
        if (Flags & object::BasicSymbolRef::SF_Undefined)
            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/vchuravy/src/julia/src/jitlayers.cpp:462:9: note: Taking false branch
        if (Flags & object::BasicSymbolRef::SF_Undefined)
        ^
/home/vchuravy/src/julia/src/jitlayers.cpp:464:13: note: Assuming the condition is false
        if (!(Flags & object::BasicSymbolRef::SF_Exported))
            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/vchuravy/src/julia/src/jitlayers.cpp:464:9: note: Taking false branch
        if (!(Flags & object::BasicSymbolRef::SF_Exported))
        ^
/home/vchuravy/src/julia/src/jitlayers.cpp:467:9: note: '?' condition is true
        assert(NameOrError);
        ^
/usr/include/assert.h:90:7: note: expanded from macro 'assert'
     (static_cast <bool> (expr)                                         \
      ^
/home/vchuravy/src/julia/src/jitlayers.cpp:469:20: note: Calling 'LegacyIRCompileLayer::findSymbolIn'
        auto Sym = JIT.CompileLayer.findSymbolIn(H, Name.str(), true);
                   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/vchuravy/builds/julia-llvm10/usr/include/llvm/ExecutionEngine/Orc/IRCompileLayer.h:140:12: note: Calling 'LegacyRTDyldObjectLinkingLayer::findSymbolIn'
    return BaseLayer.findSymbolIn(K, Name, ExportedSymbolsOnly);
           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/vchuravy/builds/julia-llvm10/usr/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h:447:12: note: Assuming the condition is true
    assert(LinkedObjects.count(K) && "VModuleKey not associated with object");
           ^~~~~~~~~~~~~~~~~~~~~~
/usr/include/assert.h:90:27: note: expanded from macro 'assert'
     (static_cast <bool> (expr)                                         \
                          ^~~~
/home/vchuravy/builds/julia-llvm10/usr/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h:447:12: note: Left side of '&&' is true
/home/vchuravy/builds/julia-llvm10/usr/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h:447:5: note: '?' condition is true
    assert(LinkedObjects.count(K) && "VModuleKey not associated with object");
    ^
/usr/include/assert.h:90:7: note: expanded from macro 'assert'
     (static_cast <bool> (expr)                                         \
      ^
/home/vchuravy/builds/julia-llvm10/usr/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h:448:12: note: Calling 'LinkedObject::getSymbol'
    return LinkedObjects[K]->getSymbol(Name, ExportedSymbolsOnly);
           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/vchuravy/builds/julia-llvm10/usr/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h:168:11: note: Assuming the condition is false
      if (SymEntry == SymbolTable.end())
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/vchuravy/builds/julia-llvm10/usr/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h:168:7: note: Taking false branch
      if (SymEntry == SymbolTable.end())
      ^
/home/vchuravy/builds/julia-llvm10/usr/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h:170:53: note: Left side of '&&' is false
      if (!SymEntry->second.getFlags().isExported() && ExportedSymbolsOnly)
                                                    ^
/home/vchuravy/builds/julia-llvm10/usr/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h:170:7: note: Taking false branch
      if (!SymEntry->second.getFlags().isExported() && ExportedSymbolsOnly)
      ^
/home/vchuravy/builds/julia-llvm10/usr/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h:172:11: note: Assuming field 'Finalized' is false
      if (!Finalized)
          ^~~~~~~~~~
/home/vchuravy/builds/julia-llvm10/usr/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h:172:7: note: Taking true branch
      if (!Finalized)
      ^
/home/vchuravy/builds/julia-llvm10/usr/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h:173:16: note: Calling constructor for 'JITSymbol'
        return JITSymbol(getSymbolMaterializer(Name),
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/vchuravy/builds/julia-llvm10/usr/include/llvm/ExecutionEngine/JITSymbol.h:269:9: note: Calling move constructor for 'unique_function<llvm::Expected<unsigned long> ()>'
      : GetAddress(std::move(GetAddress)), CachedAddr(0), Flags(Flags) {}
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/vchuravy/builds/julia-llvm10/usr/include/llvm/ADT/FunctionExtras.h:197:5: note: Taking true branch
    if (!RHS)
    ^
/home/vchuravy/builds/julia-llvm10/usr/include/llvm/ADT/FunctionExtras.h:198:7: note: Returning without writing to 'this->StorageUnion.OutOfLineStorage.StoragePtr'
      return;
      ^
/home/vchuravy/builds/julia-llvm10/usr/include/llvm/ExecutionEngine/JITSymbol.h:269:9: note: Returning from move constructor for 'unique_function<llvm::Expected<unsigned long> ()>'
      : GetAddress(std::move(GetAddress)), CachedAddr(0), Flags(Flags) {}
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/vchuravy/builds/julia-llvm10/usr/include/llvm/ExecutionEngine/JITSymbol.h:269:73: note: Returning without writing to 'this->GetAddress.StorageUnion.OutOfLineStorage.StoragePtr'
      : GetAddress(std::move(GetAddress)), CachedAddr(0), Flags(Flags) {}
                                                                        ^
/home/vchuravy/builds/julia-llvm10/usr/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h:173:16: note: Returning from constructor for 'JITSymbol'
        return JITSymbol(getSymbolMaterializer(Name),
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/vchuravy/builds/julia-llvm10/usr/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h:448:12: note: Returning from 'LinkedObject::getSymbol'
    return LinkedObjects[K]->getSymbol(Name, ExportedSymbolsOnly);
           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/vchuravy/builds/julia-llvm10/usr/include/llvm/ExecutionEngine/Orc/IRCompileLayer.h:140:12: note: Returning from 'LegacyRTDyldObjectLinkingLayer::findSymbolIn'
    return BaseLayer.findSymbolIn(K, Name, ExportedSymbolsOnly);
           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/vchuravy/src/julia/src/jitlayers.cpp:469:20: note: Returning from 'LegacyIRCompileLayer::findSymbolIn'
        auto Sym = JIT.CompileLayer.findSymbolIn(H, Name.str(), true);
                   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/vchuravy/src/julia/src/jitlayers.cpp:470:9: note: '?' condition is true
        assert(Sym);
        ^
/usr/include/assert.h:90:7: note: expanded from macro 'assert'
     (static_cast <bool> (expr)                                         \
      ^
/home/vchuravy/src/julia/src/jitlayers.cpp:474:65: note: Calling 'JITSymbol::getAddress'
        JIT.LocalSymbolTable[Name] = (void*)(uintptr_t)cantFail(Sym.getAddress());
                                                                ^~~~~~~~~~~~~~~~
/home/vchuravy/builds/julia-llvm10/usr/include/llvm/ExecutionEngine/JITSymbol.h:314:12: note: Left side of '&&' is true
    assert(!Flags.hasError() && "getAddress called on error value");
           ^
/home/vchuravy/builds/julia-llvm10/usr/include/llvm/ExecutionEngine/JITSymbol.h:314:5: note: '?' condition is true
    assert(!Flags.hasError() && "getAddress called on error value");
    ^
/usr/include/assert.h:90:7: note: expanded from macro 'assert'
     (static_cast <bool> (expr)                                         \
      ^
/home/vchuravy/builds/julia-llvm10/usr/include/llvm/ExecutionEngine/JITSymbol.h:315:5: note: Taking true branch
    if (GetAddress) {
    ^
/home/vchuravy/builds/julia-llvm10/usr/include/llvm/ExecutionEngine/JITSymbol.h:316:34: note: Calling 'unique_function::operator()'
      if (auto CachedAddrOrErr = GetAddress()) {
                                 ^~~~~~~~~~~~
/home/vchuravy/builds/julia-llvm10/usr/include/llvm/ADT/FunctionExtras.h:278:9: note: Assuming the condition is false
        isInlineStorage() ? getInlineStorage() : getOutOfLineStorage();
        ^~~~~~~~~~~~~~~~~
/home/vchuravy/builds/julia-llvm10/usr/include/llvm/ADT/FunctionExtras.h:278:9: note: '?' condition is false
/home/vchuravy/builds/julia-llvm10/usr/include/llvm/ADT/FunctionExtras.h:278:50: note: Calling 'unique_function::getOutOfLineStorage'
        isInlineStorage() ? getInlineStorage() : getOutOfLineStorage();
                                                 ^~~~~~~~~~~~~~~~~~~~~
/home/vchuravy/builds/julia-llvm10/usr/include/llvm/ADT/FunctionExtras.h:142:5: note: Undefined or garbage value returned to caller
    return StorageUnion.OutOfLineStorage.StoragePtr;
    ^      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
make: *** [/home/vchuravy/src/julia/src/Makefile:379: clang-sa-jitlayers] Error 1
make: Leaving directory '/home/vchuravy/builds/julia-llvm10/src'

@Keno I see we are passing --analyzer-no-default-checks so I assume we shouldn't be running the ReturnUndefChecker? Ah but we are actively passing -analyzer-checker=core,julia.GCChecker

@vchuravy
Copy link
Sponsor Member

Running with the core checker excluded I also get the following failure:

/home/vchuravy/src/julia/src/jitlayers.cpp:358:1: error: Non-popped GC frame present at end of function
}
^
/home/vchuravy/src/julia/src/jitlayers.cpp:328:9: note: Assuming field 'invoke' is equal to NULL
    if (unspec->invoke != NULL)
        ^~~~~~~~~~~~~~~~~~~~~~
/home/vchuravy/src/julia/src/jitlayers.cpp:328:5: note: Taking false branch
    if (unspec->invoke != NULL)
    ^
/home/vchuravy/src/julia/src/jitlayers.cpp:331:17: note: Field 'invoke' is equal to NULL
    if (unspec->invoke == NULL) {
                ^
/home/vchuravy/src/julia/src/jitlayers.cpp:331:5: note: Taking true branch
    if (unspec->invoke == NULL) {
    ^
/home/vchuravy/src/julia/src/jitlayers.cpp:333:9: note: GC frame changed here
        JL_GC_PUSH1(&src);
        ^~~~~~~~~~~~~~~~~
/home/vchuravy/src/julia/src/jitlayers.cpp:335:13: note: Assuming the condition is false
        if (jl_is_method(def)) {
            ^~~~~~~~~~~~~~~~~
/home/vchuravy/src/julia/src/julia.h:1136:30: note: expanded from macro 'jl_is_method'
#define jl_is_method(v)      jl_typeis(v,jl_method_type)
                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/vchuravy/src/julia/src/julia.h:144:25: note: expanded from macro 'jl_typeis'
#define jl_typeis(v,t) (jl_typeof(v)==(jl_value_t*)(t))
                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/vchuravy/src/julia/src/julia.h:129:22: note: expanded from macro 'jl_typeof'
#define jl_typeof(v) _jl_typeof((jl_value_t*)(v))
                     ^
/home/vchuravy/src/julia/src/jitlayers.cpp:335:9: note: Taking false branch
        if (jl_is_method(def)) {
        ^
/home/vchuravy/src/julia/src/jitlayers.cpp:349:16: note: Assuming 'src' is null
        assert(src && jl_is_code_info(src));
               ^~~
/usr/include/assert.h:90:27: note: expanded from macro 'assert'
     (static_cast <bool> (expr)                                         \
                          ^~~~
/home/vchuravy/src/julia/src/jitlayers.cpp:349:20: note: Left side of '&&' is false
        assert(src && jl_is_code_info(src));
                   ^
/home/vchuravy/src/julia/src/jitlayers.cpp:349:9: note: '?' condition is false
        assert(src && jl_is_code_info(src));
        ^
/usr/include/assert.h:90:7: note: expanded from macro 'assert'
     (static_cast <bool> (expr)                                         \
      ^
/home/vchuravy/src/julia/src/jitlayers.cpp:358:1: note: Non-popped GC frame present at end of function
}
^

@yuyichao
Copy link
Contributor Author

It thinks the assert returns? Does your assert.h have that?

In checkEndFunction there doesn't seem to be a check for noreturn function and there's a return statement passed in. Any way for you to check what's going on there?

@yuyichao
Copy link
Contributor Author

yuyichao commented Aug 23, 2020

Actually I can reproduce it too if core check is disabled. It seems that the return statement was NULL which probably mean it was end of function due to a noreturn function call. I don't know why removing the core check cause this to happen but

diff --git a/src/clangsa/GCChecker.cpp b/src/clangsa/GCChecker.cpp
index 4e7d708a67..8bba1d37d2 100644
--- a/src/clangsa/GCChecker.cpp
+++ b/src/clangsa/GCChecker.cpp
@@ -727,7 +727,7 @@ void GCChecker::checkEndFunction(const clang::ReturnStmt *RS,
     C.addTransition(State);
   if (!C.inTopFrame())
     return;
-  if (C.getState()->get<GCDepth>() > 0)
+  if (RS && C.getState()->get<GCDepth>() > 0)
     report_error(C, "Non-popped GC frame present at end of function");
 }

seems to fix it and also seems correct in general (there are other cases where a NULL is passed here and this just seem to be the only case where it actually trigger an error...)

Ref https://github.com/llvm/llvm-project/blame/8f1156a7d004d97e9f75484a00dc4278698fd8ea/clang/lib/StaticAnalyzer/Core/CoreEngine.cpp#L243-L256

Ref llvm/llvm-project@2f169e7
Ref llvm/llvm-project@6b85f8e

Also handle noreturn function call in GC frame counting,
and remove some old code for LLVM <= 7
@yuyichao
Copy link
Contributor Author

This is not user facing code and I believe there shouldn't be regression on LLVM 9 at least so I'll merge soon to make it easier for me to test a few other changes.

I also removed some old code for LLVM <= 7 support.

It also seems that the core analyzer might be responsible for handling no-return function in general. Even with the NULL return statement case factored out there are other related issues from the caller of the function. Still not sure why the core analyzer complains what it does. I have a non-asserting and release build of LLVM maybe that makes a difference?

@yuyichao yuyichao merged commit 4ed484f into master Aug 24, 2020
@yuyichao yuyichao deleted the yyc/llvm10 branch August 24, 2020 02:32
@vchuravy
Copy link
Sponsor Member

I am just running the Yggdrasil build, so that is also non-asserting and release. It's weird that we are also seeing it on CI otherwise I might blame a header difference.

simeonschaub pushed a commit to simeonschaub/julia that referenced this pull request Aug 29, 2020
Ref llvm/llvm-project@2f169e7
Ref llvm/llvm-project@6b85f8e

Also handle noreturn function call in GC frame counting,
and remove some old code for LLVM <= 7
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants