julia make fails with newest mingw (4.8.1) due to not passing the correct parameters to openblas make

[papamarkou]

It is mentioned in README.windows.md that Julia does not make properly with gcc<4.6 due to OpenBLAS problems. Nevertheless, it turns out that the issue persists even when one uses the newest version (4.8.1) of mingw-gcc (x86_64-w64-mingw32-gcc) for cross-compilation. For this reason, I git-cloned OpenBLAS separately, and the issue now has been resolved by replacing the OpenBLAS make with

make CC=x86_64-w64-mingw32-gcc FC=x86_64-w64-mingw32-gfortran BINARY=64 HOSTCC=gcc

See

OpenMathLib/OpenBLAS#226

for more details re the OpenBLAS proper compilation with the newest mingw-gcc compiler.

The reason I opened this Julia issue is because I want to see how the Julia makefile can be adapted so as to invoke the OpenBLAS make with the above appropriate parameters.

P.S. On a second note, I have notes for cross-compiling Julia on Arch Linux for Windows, in case you would like to extend the cross-compilation instructions in README.windows.md for Linux OS other than Ubuntu.

[papamarkou]

Cross-compilation works after changing the relevant OPENBLAS_BUILD_OPTS environment variable in deps/Makefile. As a temporary quick fix I added

OPENBLAS_BUILD_OPTS += HOSTCC=gcc

to make it work on my OS.

I don't submit a pull request because I don't know if there is a consensus to make cross-compilation compatible with as many systems as possible at this stage; to make the above change permanent one would have to add the above statement between lines 659-692 of deps/Makefile, and more specifically would have to amend accordingly the control flow in lines 683-692.

[vtjnash]

Nevertheless, it turns out that the issue persists even when one uses the newest version (4.8.1) of mingw-gcc (x86_64-w64-mingw32-gcc) for cross-compilation

On older gcc, the issue was that the code compiled fine, but failed the test suite for numerical issues. The newer version appears to be just missing a compile flag? Thanks for this detective work!

OPENBLAS_BUILD_OPTS += HOSTCC=gcc

I expect this should work on all OS cross-build combinations. I can add it later today.

P.S. On a second note, I have notes for cross-compiling Julia on Arch Linux for Windows, in case you would like to extend the cross-compilation instructions in README.windows.md for Linux OS other than Ubuntu.

Sure, can you submit a pull request?

[papamarkou]

Yes, I think it is just the compile flag that is missing, but to be able to answer your question, will need to do some more digging. After the openblas make finished successfully, the Julia make carried on for a while until I run into the following error:

CC src/jltypes.o                                                                                       
CC src/gf.o                                                                                                                                                                                                                              
FLISP src/julia_flisp.boot                                                                                                                                                                                                               
/bin/sh: ./flisp/flisp: cannot execute binary file                                                                                                                                                                                           
make[2]: *** [julia_flisp.boot] Error 126                                                                                                                                                                                                    
make[1]: *** [julia-release] Error 2                                                                                                                                                                                                         
make: *** [release] Error 2

I am aware that this new error has been reported before here:

#3153

So I will have to try out the solution proposed in the above thread in order to see whether the rest of the Julia cross-compilation completes successfully. I will do this and will also submit the pull request for the cross-compilation of Windows on Arch within the next couple of days.

[papamarkou]

I git-cloned Julia to cross-compile from scratch; I don't know if I should submit this as a separate issue (and also don't know if it implicitly relates to the OpenBLAS issue, which has been resolved after making the above change to OPENBLAS_BUILD_OPTS). At the final link stage I get the following error, which I haven't managed to resolve:

CC src/jltypes.o
CC src/gf.o
CC src/support/hashing.o
CC src/support/timefuncs.o
CC src/support/dblprint.o
CC src/support/ptrhash.o
CC src/support/operators.o
CC src/support/utf8.o
CC src/support/ios.o
CC src/support/htable.o
CC src/support/bitvector.o
CC src/support/int2str.o
CC src/support/libsupportinit.o
CC src/support/arraylist.o
CC src/support/asprintf.o
CC src/support/wcwidth.o
CC src/support/_setjmp.win64.o
CC src/support/_longjmp.win64.o
LINK src/support/libsupport.a
CC src/flisp/flisp.o
CC src/flisp/builtins.o
CC src/flisp/string.o
CC src/flisp/equalhash.o
CC src/flisp/table.o
CC src/flisp/iostream.o
CC src/flisp/julia_extensions.o
LINK src/flisp/libflisp.a
CC src/flisp/flmain.o
CC src/flisp/flisp
./flisp: ./flisp: cannot execute binary file
make[3]: *** [flisp] Error 126
make[2]: *** [flisp/libflisp.a] Error 2
make[1]: *** [julia-release] Error 2
make: *** [release] Error 2

Any ideas what may be causing this error? I may have to change sth in src/flisp/Windows.mk...

[vtjnash]

I think this means that you can't execute windows binaries without calling wine. That would have been a kernel compile option. I think this can be easily fixed by replacing the call to ./flisp ... with $(call spawn,flisp) ...

[vtjnash]

The Windows.mk file is for the windows nmake compiler. The cross-compile works entirely through gmake Makefile's

[papamarkou]

Great, the Julia make on Arch for Windows using mingw has completed successfully, after changing lines 56-61 in src/Makefile from

julia_flisp.boot.inc: julia_flisp.boot flisp/libflisp.a
    $(QUIET_FLISP) ./flisp/flisp ./bin2hex.scm < $< > $@

julia_flisp.boot: julia-parser.scm julia-syntax.scm \
    match.scm utils.scm jlfrontend.scm mk_julia_flisp_boot.scm flisp/libflisp.a
    $(QUIET_FLISP) ./flisp/flisp ./mk_julia_flisp_boot.scm

to

julia_flisp.boot.inc: julia_flisp.boot flisp/libflisp.a
    $(QUIET_FLISP) $(call spawn,./flisp/flisp) ./bin2hex.scm < $< > $@

julia_flisp.boot: julia-parser.scm julia-syntax.scm \
    match.scm utils.scm jlfrontend.scm mk_julia_flisp_boot.scm flisp/libflisp.a
    $(QUIET_FLISP) $(call spawn,./flisp/flisp) ./mk_julia_flisp_boot.scm

Would it be useful to make this change permanent?

After make, the commands mkdir dist-extras && make win-extras and make dist also completed successfully. I will now write down the steps for doing the mingw cross-compilation on Arch and will submit the pull request to extend the installation instructions in README.windows.md. Thanks for the flisp tip Jameson.

[papamarkou]

I submitted the pull request with the updated instructions to include Arch cross-compilation. May I close this issue?