Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump dependencies to fix ppc64le page size issues #34082

Merged
merged 8 commits into from
Dec 14, 2019
Merged

Conversation

staticfloat
Copy link
Sponsor Member

@staticfloat staticfloat commented Dec 12, 2019

This PR is to see if we can't get PPC64le bootstrapping again.

Dependencies bumped so far:

  • GMP
  • MPFR
  • MbedTLS
  • libssh2
  • libgit2
  • libCURL

@staticfloat staticfloat changed the title Bump MPFR and GMP to fix ppc64le page size issues Bump dependencies to fix ppc64le page size issues Dec 12, 2019
@staticfloat staticfloat reopened this Dec 12, 2019
@staticfloat staticfloat reopened this Dec 12, 2019
@ararslan ararslan added domain:building Build system, or building Julia or its dependencies external dependencies Involves LLVM, OpenBLAS, or other linked libraries system:powerpc PowerPC labels Dec 12, 2019
@ViralBShah
Copy link
Member

Would be good to perhaps add any other stdlib binaries that need to be prepared for JLLs here too. I think we are very close.

@giordano
Copy link
Contributor

giordano commented Dec 12, 2019

We've recently rebuilt libuv and libunwind, is it worth bumping them as well here?

@vchuravy
Copy link
Sponsor Member

Seems almost ready except for FreeBSD and Mac OSX?

https://build.julialang.org/#/builders/45/builds/7841 is the first successful PPC package build.

FreeBSD

LoadError("sysimg.jl", 16, LoadError("/usr/home/julia/buildbot/worker/package_freebsd64/build/usr/share/julia/stdlib/v1.4/SuiteSparse/src/SuiteSparse.jl", 24, LoadError("/usr/home/julia/buildbot/worker/package_freebsd64/build/usr/share/julia/stdlib/v1.4/SuiteSparse/src/umfpack.jl", 67, ErrorException("error compiling top-level scope: could not load library \"libsuitesparse_wrapper\"\nShared object \"libopenblas64_.0.3.7.so\" not found, required by \"libcholmod.so.3\""))))

Mac OSX:

LoadError("sysimg.jl", 16, LoadError("/Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.4/SuiteSparse/src/SuiteSparse.jl", 24, LoadError("/Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.4/SuiteSparse/src/umfpack.jl", 67, ErrorException("error compiling top-level scope: could not load library \"libsuitesparse_wrapper\"\ndlopen(libsuitesparse_wrapper.dylib, 1): Library not loaded: @rpath/libopenblas64_.0.3.7.dylib\n  Referenced from: /Users/julia/buildbot/worker/package_macos64/build/usr/lib/./libcholmod.3.0.13.dylib\n  Reason: image not found"))))

@giordano
Copy link
Contributor

MacOS should be fixed by JuliaPackaging/Yggdrasil#285

@vchuravy
Copy link
Sponsor Member

@staticfloat how do you get the hashes for all platforms? Is there an automated way of going about it?

@giordano
Copy link
Contributor

The hashes of the tarballs?

@vchuravy
Copy link
Sponsor Member

The hashes of the tarballs?

Yeah. Bumping SuiteSparse was not enough for FreeBSD. It fails with the same error.

@giordano
Copy link
Contributor

Yeah

He maybe uses something smarter here, but you can get the hashes for tarballs by generating the build.jl with https://github.com/JuliaPackaging/Yggdrasil/blob/master/README.md#binaryproviderjl

Bumping SuiteSparse was not enough for FreeBSD. It fails with the same error.

Yes, I wasn't expecting that would have fixed FreeBSD 😞

@staticfloat
Copy link
Sponsor Member Author

how do you get the hashes for all platforms? Is there an automated way of going about it?

Yeah, in this case you might run ./contrib/refresh_bb_tarballs.sh suitesparse and it will download all the different tarballs, checking their hashes, (and if there is no hash for that version creating one).

@staticfloat
Copy link
Sponsor Member Author

JuliaPackaging/Yggdrasil#287

@vchuravy vchuravy marked this pull request as ready for review December 14, 2019 10:24
@vchuravy
Copy link
Sponsor Member

This is looking good, if CI is green I will merge (and squash) this.

https://build.julialang.org/#/builders/45/builds/7879

@vchuravy vchuravy merged commit 4470c94 into master Dec 14, 2019
@vchuravy vchuravy deleted the sf/fix_ppc64le_deps branch December 14, 2019 19:44
@timholy
Copy link
Sponsor Member

timholy commented Dec 15, 2019

On a0b67dc (the commit before this one), I can build Julia. But with this commit I get

$ make
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   501  100   501    0     0   2517      0 --:--:-- --:--:-- --:--:--  2505
100 1423k  100 1423k    0     0  1902k      0 --:--:-- --:--:-- --:--:-- 1902k
===============================================================================
  ERROR: sha512 checksum failure on SuiteSparse.v5.4.0-2.x86_64-linux-gnu.tar.gz, should be:
      841050c51b5965dc3ccb5b933df09d33e5912b435c39cf67098aed4d30386cf1
      9825433aea2b2af4668f7358f49ddc178444831eeeeb996bb35328c51bb2441d
  But `sha512sum /home/tim/src/julia-master/deps/srccache/SuiteSparse.v5.4.0-2.x86_64-linux-gnu.tar.gz | awk '{ print $1; }'` results in:
      7b49f0bd1a20104df6993e949ab0e8aa25b7ea64250a726e9c1b0312debdbb87
      c949e4449790de4eaffd072a671da32da8a5301ee62b589ce05ea350f65a3e3e
  This can happen due to bad downloads or network proxies, please check your
  network proxy/firewall settings and delete
  /home/tim/src/julia-master/deps/srccache/SuiteSparse.v5.4.0-2.x86_64-linux-gnu.tar.gz
  to force a redownload when you are ready
===============================================================================
/home/tim/src/julia-master/deps/suitesparse.mk:144: recipe for target '/home/tim/src/julia-master/usr/manifest/suitesparse' failed
make[1]: *** [/home/tim/src/julia-master/usr/manifest/suitesparse] Error 2
Makefile:60: recipe for target 'julia-deps' failed
make: *** [julia-deps] Error 2

StefanKarpinski added a commit that referenced this pull request Dec 15, 2019
vchuravy added a commit that referenced this pull request Dec 16, 2019
* Bump `MPFR`
* Bump `GMP`
* Bump `MbedTLS`
* Bump `LibSSH2`
* Update `LibGit2`
* Bump `LibCURL`
* Bump SuiteSparse to 5.4.0-2, also mark it as no longer libgfortran-expanded

Co-authored-by: Elliot Saba <[email protected]>
Co-authored-by: Valentin Churavy <[email protected]>
vchuravy added a commit that referenced this pull request Dec 16, 2019
* Bump `MPFR`
* Bump `GMP`
* Bump `MbedTLS`
* Bump `LibSSH2`
* Update `LibGit2`
* Bump `LibCURL`
* Bump SuiteSparse to 5.4.0-2, also mark it as no longer libgfortran-expanded

Co-authored-by: Elliot Saba <[email protected]>
Co-authored-by: Valentin Churavy <[email protected]>
KristofferC pushed a commit that referenced this pull request Apr 11, 2020
* Bump `MPFR` 
* Bump `GMP` 
* Bump `MbedTLS`
* Bump `LibSSH2`
* Update `LibGit2`
* Bump `LibCURL`
* Bump SuiteSparse to 5.4.0-2, also mark it as no longer libgfortran-expanded
KristofferC pushed a commit that referenced this pull request Apr 11, 2020
KristofferC pushed a commit that referenced this pull request Apr 11, 2020
* Bump `MPFR`
* Bump `GMP`
* Bump `MbedTLS`
* Bump `LibSSH2`
* Update `LibGit2`
* Bump `LibCURL`
* Bump SuiteSparse to 5.4.0-2, also mark it as no longer libgfortran-expanded

Co-authored-by: Elliot Saba <[email protected]>
Co-authored-by: Valentin Churavy <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
domain:building Build system, or building Julia or its dependencies external dependencies Involves LLVM, OpenBLAS, or other linked libraries system:powerpc PowerPC
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants