-
-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Repeated calls to LibGit2.fetch segfaults on FreeBSD 11.1 #23328
Comments
Running it in GDB, I'm seeing a lot of dwarf errors regarding "wrong version in compilation unit header," as well as
at the end. Full log here: https://gist.github.com/ararslan/a9694acc54da13f633edde6aa7230a59 It also left a core file in the package directory where the tests were running. Examining that with GDB, I get
|
your gdb version is too old,
please try out devel/gdb?
…On Sat, Aug 19, 2017 at 12:00:57AM +0000, Alex Arslan wrote:
Running it in GDB, I'm seeing a lot of dwarf errors regarding "wrong version in compilation unit header," as well as
```
Program received signal SIGSEV, Segmentation fault.
0x0000000801571c19 in ?? () /usr/local/lib/gcc5/libgcc_s.so.1
```
at the end. Full log here: https://gist.github.com/ararslan/a9694acc54da13f633edde6aa7230a59
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#23328 (comment)
--
Iblis Lin
林峻頤
|
Wow, the system GDB is way out of date, no wonder they're going to remove it. Okay, I tried again with the ports GDB. The output is much more informative, but it looks like it's stopping on something else; the test program doesn't get nearly as far before stopping, and it doesn't appear to be hitting a segfault in GDB. The full log is here: https://gist.github.com/ararslan/4347c22e925e98f5fcacda5b438854a9. Could be I'm misusing GDB somehow, so if anything about the session in the log looks fishy let me know and I can try it another way. |
Using LLDB rather than GDB looks like it hits the right thing: https://gist.github.com/ararslan/acc5e587400affd93dff5e41f5dc1cda. (System LLDB, 4.0.0 in FreeBSD 11.1.) |
Looks like it's this line that's hitting it: https://github.com/JuliaLang/julia/blob/master/test/pkg.jl#L298 @test_warn "INFO: Package Example: skipping update (pinned)..." Pkg.update() I haven't been able to minimally reduce the issue yet though. |
Found a workaround, can you confirm?:
It appears that this option is enabled in FreeBSD 11.1-Release by default. |
Interesting, it does appear that |
Edit: seems unrelated -- this commit enable it by default https://svnweb.freebsd.org/base?view=revision&revision=320317 |
Okay, even better LLDB backtrace: https://gist.github.com/ararslan/62d5dfb03b529a56dce0ab5d239685d8. In particular, it shows libgit2 calls in Edit: Buuuuuuuuut that call is in the wrong thread. :/ Thread 6 in the above gist hits the SIGSEGV. |
Similar issue? https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=221127#c12 I'm going to give |
I doubt it's an issue with conflicting libgcc_s between Clang and GCC since we have the build system set up on FreeBSD to link everything to GCC's libgcc_s. That's what I implemented in #21788. It must be an issue with the stack guard setting, since the tests pass with it disabled and segfault with it enabled. |
on
I think it's releated to r320317. It does change something besides setting stack guard as default; 11.0-RELEASE doesn't include this patch, so |
Excellent catch! |
Ok, maybe we should create an minimal example, and send it to FreeBSD CURRENT mailing list. |
I'm not sure the problem is libgit2; in the LLDB backtrace that contains libgit2 calls, the libgit2 calls are in a different thread than the one that hits the SIGSEGV. Do you know how to reproduce this minimally? I'm still trying to figure out what in Julia is actually triggering it. |
😖 |
Which tests did you have to remove in test/pkg.jl to get the tests passing? Can you post a diff? |
please checkout this: https://gist.github.com/iblis17/e2199735aee9673585da8aa48e5d4984 In the original |
For record, segfault happened on -CURRENT with stack guard enabled, also.
|
another reproduce steps:
for i ∈ 1:100
info(i)
LibGit2.fetch(LibGit2.GitRepo("./repo"))
end
|
Good sleuthing! If you write the same thing in C using the functions from libgit2, do you also get a segfault? Also, why is the |
I checked it again, |
I was able to reduce it to a single import Base.LibGit2: GitRepo, GitRemote, RemoteCallbacks, CredentialPayload,
StrArrayStruct, FetchOptions, get, credentials_cb
repo = GitRepo("./repo")
rmt = get(GitRemote, repo, "origin")
fo = FetchOptions(callbacks=RemoteCallbacks(credentials_cb(), CredentialPayload()))
for i = 1:100
info(i)
ccall((:git_remote_fetch, :libgit2), Cint,
(Ptr{Void}, Ptr{StrArrayStruct}, Ptr{FetchOptions}, Cstring),
rmt.ptr, C_NULL, Ref(fo), "hi")
end
close(rmt)
close(repo) For me it consistently faults on the sixth call, as it is in your example output above. |
Is it possible that one of the structs has the wrong specification? |
I compared our implementations in base/libgit2/types.jl to the documentation of the corresponding types in libgit2 and they seem to match as far as I can tell, but there may be some subtle difference that I'm missing. |
I'm going to call this a Julia bug since I haven't been able to reproduce it with C or Rust's libgit2 bindings. |
@omus has noted that this seems only to occur when using multiple cores. That is, setting |
Hm, setting |
My guess that something happens during the |
Why would something inside |
Okay, updated LLDB output on latest master with the above |
LLDB backtrace on FreeBSD 12.0-CURRENT with |
Plot twist: libgit2 may be a red herring. Setting |
I tried copying my |
Using the system curl with |
Never mind, it does—I just had to rebuild libgit2 and mbedTLS. |
I'm not getting a segfault with a stock build of Julia on FreeBSD 12.0-CURRENT, built from source at |
I just tried current master on FreeBSD 11.1 and it didn't segfault. I'm doing a |
Neither Iblis nor I can reproduce this on 11.1 with current master, so I'm going to close this and call it resolved. I have no idea what changed in Base that made it work, though I might bisect it out of curiosity. |
I upgraded my FreeBSD box from 11.0 (which is what the FreeBSD CI workers are running) to 11.1, and now I'm consistently getting segfaults in the Pkg tests:
Version info:
cc @iblis17
The text was updated successfully, but these errors were encountered: