Add libgit2 dependency to build system

[tkelman]

This is pretty much #7339 with all changes to base removed. And I'm changing the way I'm handling flags and compiler names with cmake in the last commit.

[jakebolewski]

Do we want to build the tests by default? Otherwise we should add -DCLAR_BUILD=OFF here.

[tkelman]

We can turn them off to save build time, I think I have them commented out (running them that is)

[jakebolewski]

This builds fine for me on OS X / Linux. @tkelman can you check if the version check is correct for Windows source builds? If the Makefile changes are acceptable I would say this is good to merge.

[tkelman]

there was a conflict in the gitignore, so I rebased it

[jakebolewski]

If CMake is now necessary for the default source build then the README should be updated.

[tkelman]

yes good point, will do

[tkelman]

note for @quinnj or @ihnorton or others doing MSYS2 Windows source builds, I've found installing cmake through pacman messes with MSYS2 paths too much (it's in the mingw packages group rather than msys) and it seems to be easier to use the cmake.org installer. Have to either add the folder to your path or set its location in Make.user (guess which one I prefer).

[tkelman]

I'm going to guess this is unrelated, but what's up with

    From worker 2:       * linalg1
    From worker 3:       * linalg2
warning: stack corruption detected
signal (11): Segmentation fault: 11
jl_type_intersect at /Users/travis/build/JuliaLang/julia/src/jltypes.c:788
jl_type_intersection_matching at /Users/travis/build/JuliaLang/julia/src/jltypes.c:1389
jl_type_intersection at /Users/travis/build/JuliaLang/julia/src/jltypes.c:1092
Worker 2 terminated.
ERROR: ProcessExitedException()
 in wait at /private/tmp/julia/lib/julia/sys.dylib (repeats 2 times)
 in wait_full at /private/tmp/julia/lib/julia/sys.dylib
 in remotecall_fetch at multi.jl:676
 in remotecall_fetch at multi.jl:681
 in anonymous at task.jl:1450
while loading /private/tmp/julia/share/julia/test/runtests.jl, in expression starting on line 41

on osx travis?

[quinnj]

Thanks for the heads up @tkelman.

[StefanKarpinski]

So what's the status on this?

[tkelman]

It builds, which I think is the point here. I had to rebase a day or two ago to fix a conflict. If you're happy with my doc additions, I think it's time to merge and get started rewriting Pkg.

[tkelman]

And I'm pretty sure I resolved the cmake compiler name pickiness that @vtjnash was objecting to in #7339 in a way that shouldn't break anything. (6300cbe - @Keno this shouldn't break your libcxx stuff either, but you might want a heads-up)

[vtjnash]

Yes, I like your solution quite a lot

The version test in libgit2.jl seems like it may cause issues for system packagers. I hope we don't have that strict of a requirement on the version.

It might be good to mention that msys cmake will get confused, but it sounds like we may need to switch to a Cygwin cross-compile anyways?

Lgtm to merge

[vtjnash]

Can you enable the code so that 'make libgit2-test' does something? Or why is that commented out?

[jakebolewski]

We may not need to use one of the bugfix releases, but we definitely need to use the v0.21 release. Much of the library was rewritten during the last release cycle and many api's changed. I think we should use the bugfix release for now as it would be annoying to run into solved problems which have been back-ported. Also we should bump the version to v0.21.2 which was released a couple days ago.

[tkelman]

@vtjnash I don't remember exactly what was going wrong with the libgit2 self-checks, I think they may have been failing on one or more platforms? We can also switch from cmake --build to just recursive make so any parallel -j flags propagate to libgit.

[tkelman]

we may need to switch to a Cygwin cross-compile anyways?

That depends on whether we can figure out what's wrong in LLVM 3.5.0 with the mingw-builds compiler version we've been using with the msys2 build. We can't use the mingw compilers from pacman, at least for 32 bit, since they're using dwarf exceptions. I don't think the msys2 pacman version of cmake is unable to build this, but it does mess with the /mingw64 paths because it's not the posix-style cmake.

[vtjnash]

Can we give it a version range then (v0.21+ perhaps)?

[jakebolewski]

The test doesn't check the patch number, only the major / minor version numbers.

[vtjnash]

Ok, but it looks like Ubuntu has 0.19 or 0.20 right now https://packages.ubuntu.com/source/utopic/libgit2

[jakebolewski]

v0.21 was released in June. I don't think it is possible to support any older version, merge support was added in v0.21.

[vtjnash]

It should be ok, since Julia 0.4 wouldn't be in Ubuntu till next spring or fall anyways. Just thought it might be worth mentioning, although I would also support getting this merged now so we can start to see what issues it might bring with it

[tkelman]

Don't think it has to be perfect in this pr, we can keep working on it on master

[jakebolewski]

0.21 was a breaking release as they are trying to get the api's settled for v1.0. There should not be as much api breakage going forward.

[jakebolewski]

Bump. Can we merge this? I have time to work on this this week.

[staticfloat]

+1 for just merging and dealing with it this week

[StefanKarpinski]

Yeah, I'm in favor. @tkelman, your call.

[tkelman]

Well if you're gonna put it that way...

[staticfloat]

@tkelman

[ihnorton]

s'guud

On Mon, Nov 3, 2014 at 4:34 PM, Elliot Saba [email protected]
wrote:

@tkelman https://github.com/tkelman [image: tkelman right now]
https://camo.githubusercontent.com/7e12c0f0b76f46a1b6974c6a5cf96c1b385e6a22/687474703a2f2f7777772e717569636b6d656d652e636f6d2f696d672f34372f343734653534383065383464303135343865643931663539663032623439336263353833643639363363653961613263366631373933333133656166343438372e6a7067

—
Reply to this email directly or view it on GitHub
#8820 (comment).

[StefanKarpinski]

Ah, yes. Finally!

[tkelman]

will need ppa-packaging, and bottling once the more pressing osx-travis blockers are re-bottled

edit: we could borrow from / use https://launchpad.net/~presslabs/+archive/ubuntu/gitfs-dev/+packages for the ppa, and homebrew already bottles libgit2 for 10.8 and newer

[quinnj]

Finally updating to master tonight and everything built without a hitch. Really big thanks and kudos to @tkelman for the great work here.

[tkelman]

@quinnj Good to hear, you're very welcome, but getting it into the build system is just the start. And we haven't had any successful binaries uploaded since this merge. But that's mostly due to the low bus factor right now on our ability to build official binaries.

[tkelman]

@staticfloat oh, whoops, looks like getting rid of the compile instructions for git broke the mac binaries. Sorry, my bad! Does the mac app still need to build git from source? If so I'll put it back.

https://buildbot.e.ip.saba.us:8010/builders/package_osx10.9/builds/278/steps/shell_6/logs/stdio

julia/contrib/mac/app/Makefile

Line 18 in bb628af

make -C ../../../deps install-git

[staticfloat]

We do need to bundle git with the mac dmgs, since without that, we don't
necessarily have git.
-E

On Sun, Nov 9, 2014 at 1:30 PM, Tony Kelman [email protected]
wrote:

@staticfloat https://github.com/staticfloat oh, whoops, looks like
getting rid of the compile instructions for git broke the mac binaries.
Sorry, my bad! Does the mac app still need to build git from source? If so
I'll put it back.

https://buildbot.e.ip.saba.us:8010/builders/package_osx10.9/builds/278/steps/shell_6/logs/stdio

julia/contrib/mac/app/Makefile

Line 18 in bb628af

make -C ../../../deps install-git

—
Reply to this email directly or view it on GitHub
#8820 (comment).

[tkelman]

Okay I'll put it back, sorry I missed it - don't think I've ever looked into contrib/mac before. Can never keep track of what Apple installs by default vs what comes in the command line tools. We can get rid of it for real once libgit is fully ready, same as for the Windows binaries.