Update Mozilla CA certificate store to latest (05-15-2019) for libgit2 SSL.

[mikhail-j]

@staticfloat
The curl project has updated the Mozilla CA certificate store earlier today (05/15/2019).

This update coincides with CA certificate list changes to Mozilla NSS 3.43.

5 CA certificates were added:

• emSign Root CA - G1
• emSign ECC Root CA - G3
• emSign Root CA - C1
• emSign ECC Root CA - C3
• Hongkong Post Root CA 3

On a different note, compiling Julia has become rather difficult on older glibc versions (#31555 and #31605) due to BinaryBuilder's recent integration with the makefiles.

Are there any plans to add a build option to compile from scratch for both the application and the tests?

[ViralBShah]

You can build from scratch (there's a flag to disable BB).

[staticfloat]

Luckily, I have recently rebuilt a large chunk of the BinaryBuilder compilers to address that exact issue; so we will have better support for these old linkers and such once that new BinaryBuilder version is released and we update all of the built tarballs with those new compiler versions. I estimate 3-5 weeks before everything is released and updated, in the meantime you can set USE_BINARYBUILDER=0 to disable that behavior, or if you're only having a problem with libuv (which could be the case, as very few of our binary dependencies want to link against static libraries, most of them do dynamic linking, which don't have these same incompatibilities) you can set USE_BINARYBUILDER_LIBUV=0 to keep the lion's share of the build time improvements you get from using BB-built tarballs.

[mikhail-j]

@ViralBShah @staticfloat

Thank you for the helpful information.

To clarify, here's my build environment:

• SUSE SLES SP4
• glibc version 2.11.3
• GNU binutils version 2.25.0

I did try to use USE_BINARYBUILDER=0 in Make.user as specified in build.md. But this option was ignored during compilation:

/usr/bin/ld: libuv_la-uv-common.o: access beyond end of merged section (4974)
/usr/bin/ld: libuv_la-uv-common.o: access beyond end of merged section (5695)
/usr/bin/ld: libuv_la-uv-common.o: access beyond end of merged section (5686)
/usr/bin/ld: libuv_la-uv-common.o: access beyond end of merged section (5446)
/usr/bin/ld: libuv_la-uv-common.o: access beyond end of merged section (4833)
/usr/bin/ld: libuv_la-uv-common.o: access beyond end of merged section (3254)
/usr/bin/ld: libuv_la-uv-common.o: access beyond end of merged section (6232)
/usr/bin/ld: libuv_la-uv-common.o: access beyond end of merged section (6837)
/usr/bin/ld: libuv_la-uv-common.o: access beyond end of merged section (4011)
/usr/bin/ld: libuv_la-uv-common.o: access beyond end of merged section (3967)
/usr/bin/ld: libuv_la-uv-common.o: access beyond end of merged section (4912)
/usr/bin/ld: libuv_la-uv-common.o: access beyond end of merged section (5048)
/usr/bin/ld: libuv_la-uv-common.o: access beyond end of merged section (6811)
/usr/bin/ld: libuv_la-uv-common.o: access beyond end of merged section (6800)
/usr/bin/ld: libuv_la-uv-common.o: access beyond end of merged section (5446)
/usr/bin/ld: libuv_la-uv-common.o: access beyond end of merged section (4833)
/usr/bin/ld: libuv_la-uv-common.o: access beyond end of merged section (3254)
/usr/bin/ld: libuv_la-uv-common.o: access beyond end of merged section (6232)
/usr/bin/ld: libuv_la-uv-common.o: access beyond end of merged section (6837)
/usr/bin/ld: libuv_la-uv-common.o: access beyond end of merged section (5763)
/usr/bin/ld: libuv_la-uv-common.o: access beyond end of merged section (5446)
/usr/bin/ld: libuv_la-uv-common.o: access beyond end of merged section (4833)
/usr/bin/ld: libuv_la-uv-common.o: access beyond end of merged section (3254)
/usr/bin/ld: libuv_la-uv-common.o: access beyond end of merged section (6232)
/usr/bin/ld: libuv_la-uv-common.o: access beyond end of merged section (6837)
/usr/bin/ld: libuv_la-uv-common.o: access beyond end of merged section (4256)
/usr/bin/ld: libuv_la-uv-common.o: access beyond end of merged section (5638)
/usr/bin/ld: $HOME/julia/usr/lib/libuv.a(libuv_la-uv-common.o)(.debug_info+0x10cc): reloc against `.debug_str': error 2
/usr/bin/ld: final link failed: Nonrepresentable section on output
collect2: error: ld returned 1 exit status
make[2]: *** [$HOME/julia/src/flisp/flisp] Error 1
make[1]: *** [flisp/libflisp.a] Error 2
make: *** [julia_flisp.boot.inc.phony] Error 2

However, my compilation was successful when I passed USE_BINARYBUILDER=0 directly as an argument to make:

make -j N USE_BINARYBUILDER=0

The documentation on USE_BINARYBUILDER=0 did not specify if it was used in make test.

So, I tried executing make test by itself and this resulted in a broken julia executable:

$HOME/julia> ./usr/bin/julia
./usr/bin/julia: /lib64/libpthread.so.0: version `GLIBC_2.12' not found (required by $HOME/julia/usr/bin/../lib/libLLVM-6.0.so)

After further testing, I successfully ran the make tests when I specified USE_BINARYBUILDER=0 as an argument to make test:

make USE_BINARYBUILDER=0 test

As a result, I was able to successfully compile my nightly build of Julia with glibc 2.11.3.

I learned that setting USE_BINARYBUILDER_LIBUV=0 alone was insufficient for successful compilation.

I understand that BinaryBuilder build time improvements can shorten the runtime of Julia continuous integration testing.

Analogous to the Ship of Theseus, BinaryBuilder is essentially downloading official binaries piece by piece. However, the default BinaryBuilder flag conflicts with the reasons why I compile Julia rather than downloading an official binary. I have to compile Julia because no official binaries exist to run on my system architecture.

Is there a plan to set USE_BINARYBUILDER=0 as default and USE_BINARYBUILDER=1 in the continuous integration testing?

[staticfloat]

I did try to use USE_BINARYBUILDER=0 in Make.user as specified in build.md

Unfortunately, you currently need to use override USE_BINARYBUILDER=0, due to a Makefile bug that hasn't been fixed yet (but I just opened a new PR to do this). As you have surmised, setting USE_BINARYBUILDER=0 on the command line works, because it is in effect doing the same thing as writing override USE_BINARYBUILDER=0 in Make.inc.

As a result, I was able to successfully compile my nightly build of Julia with glibc 2.11.3.

While that is a pretty ancient glibc (you must have some highly patched compilers that both work with glibc 2.11 and also are able to compile Julia; I tip my hat to whoever is maintaining your distro) the issues you're seeing with libuv_la-uv-common.o are because of your binutils version. The issue you saw with libpthread.so.0 is because of your old glibc version. :)

I learned that setting USE_BINARYBUILDER_LIBUV=0 alone was insufficient for successful compilation.

Yes, because of your old glibc version, you will need to disable pretty much all BB tarball usage, because the binaries may use APIs that are available in newer versions.

I understand that BinaryBuilder build time improvements can shorten the runtime of Julia continuous integration testing.

Not only for integration testing, but for the vast majority of users building Julia from source it cuts down on both time and many build problems, as the surface area of things that can go wrong is greatly decreased.

Is there a plan to set USE_BINARYBUILDER=0 as default and USE_BINARYBUILDER=1 in the continuous integration testing?

No, for the large majority of users this is a very helpful feature. It is unfortunate that you ran into two problems (The first being binutils incompatibility, which is something that will be eventually sorted out. The second being glibc incompatibility, which is something that will most likely not be sorted out) but we have to draw the line somewhere at the range of systems that we will pour time and energy into making the default experience smooth for. We will of course still help and support users with old systems build Julia, that is something that we will always do and try to make sure things are as smooth as possible; but you may need to tweak some levers to get Julia to build on systems such as yours. In this case, the levers you will need to tweak will be setting USE_BINARYBUILDER=0, which we continue to support.

One possible thing that could help with this is to automatically disable USE_BINARYBUILDER=0 if we notice that the glibc version is too old; we may be able to parse this out via the output of ldd --version, in a similar way to how we parse out the gfortran version. Feel free to open an issue if you'd like to help us add this in.

[staticfloat]

Also, are you really able to build Julia with GCC 4.3?! I've tried with 4.6 and I ran into too many compiler bugs/incompatibilities. Are you not actually using a newer version of GCC? I think we advertise that a minimum version of 4.7 is required to get through the full build.

[mikhail-j]

@staticfloat
Thanks for informing me of your future plans for BinaryBuilder settings.

you must have some highly patched compilers that both work with glibc 2.11 and also are able to compile Julia; I tip my hat to whoever is maintaining your distro

I am not using the GCC compiler suite provided by SUSE SLES 11 SP4.

Also, are you really able to build Julia with GCC 4.3?! I've tried with 4.6 and I ran into too many compiler bugs/incompatibilities. Are you not actually using a newer version of GCC? I think we advertise that a minimum version of 4.7 is required to get through the full build.

I used GCC 4.8.5 (which I compiled with the GCC compiler provided by SUSE SLES 11) to compile Julia.