PCRE 10.20 build fails

[Keno]

cc @tkelman

/home/kfischer/julia/deps/jldownload pcre2-10.20.tar.bz2 https://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre2-10.20.tar.bz2
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   317  100   317    0     0    767      0 --:--:-- --:--:-- --:--:--   769
100 1326k  100 1326k    0     0  1675k      0 --:--:-- --:--:-- --:--:-- 9412k
/home/kfischer/julia/deps/jlchecksum pcre2-10.20.tar.bz2
`which gtar 2>/dev/null || which tar 2>/dev/null` jxf pcre2-10.20.tar.bz2
touch -c pcre2-10.20/configure
cd pcre2-10.20 && \
./configure --prefix=/home/kfischer/julia/usr --build=x86_64-pc-linux-gnu --libdir=/home/kfischer/julia/usr/lib  F77="gfortran -m64" CC="clang -m64 " CXX="clang++ -m64 " --enable-jit --includedir=/home/kfischer/julia/usr/include CFLAGS=" -O3" LDFLAGS=" "-Wl,-rpath,'/home/kfischer/julia/usr/lib'""
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether make supports nested variables... (cached) yes
checking for style of include used by make... GNU
checking for gcc... clang -m64
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether clang -m64  accepts -g... yes
checking for clang -m64  option to accept ISO C89... none needed
checking whether clang -m64  understands -c and -o together... yes
checking dependency style of clang -m64 ... gcc3
checking for ar... ar
checking the archiver (ar) interface... ar
checking for gcc... (cached) clang -m64
checking whether we are using the GNU C compiler... (cached) yes
checking whether clang -m64  accepts -g... (cached) yes
checking for clang -m64  option to accept ISO C89... (cached) none needed
checking whether clang -m64  understands -c and -o together... (cached) yes
checking dependency style of clang -m64 ... (cached) gcc3
checking how to run the C preprocessor... clang -m64  -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for int64_t... yes
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking how to print strings... printf
checking for a sed that does not truncate output... /bin/sed
checking for fgrep... /bin/grep -F
checking for ld used by clang -m64 ... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... dlltool
checking how to associate runtime and link libraries... printf %s\n
checking for archiver @FILE support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from clang -m64  object... ok
checking for sysroot... no
checking for a working dd... /bin/dd
checking how to truncate binary pipes... /bin/dd bs=4096 count=1
checking for mt... mt
checking if mt is a manifest tool... no
checking for dlfcn.h... yes
checking for objdir... .libs
checking if clang -m64  supports -fno-rtti -fno-exceptions... yes
checking for clang -m64  option to produce PIC... -fPIC -DPIC
checking if clang -m64  PIC flag -fPIC -DPIC works... yes
checking if clang -m64  static flag -static works... yes
checking if clang -m64  supports -c -o file.o... yes
checking if clang -m64  supports -c -o file.o... (cached) yes
checking whether the clang -m64  linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking whether ln -s works... yes
checking whether the -Werror option is usable... yes
checking for simple visibility declarations... yes
checking for ANSI C header files... (cached) yes
checking limits.h usability... yes
checking limits.h presence... yes
checking for limits.h... yes
checking for sys/types.h... (cached) yes
checking for sys/stat.h... (cached) yes
checking dirent.h usability... yes
checking dirent.h presence... yes
checking for dirent.h... yes
checking windows.h usability... no
checking windows.h presence... no
checking for windows.h... no
checking for an ANSI C-conforming const... yes
checking for size_t... yes
checking for bcopy... yes
checking for memmove... yes
checking for strerror... yes
checking zlib.h usability... yes
checking zlib.h presence... yes
checking for zlib.h... yes
checking for gzopen in -lz... yes
checking bzlib.h usability... no
checking bzlib.h presence... no
checking for bzlib.h... no
checking for libbz2... no
checking for the pthreads library -lpthreads... no
checking whether pthreads work without any flags... no
checking whether pthreads work with -Kthread... no
checking whether pthreads work with -kthread... no
checking for the pthreads library -llthread... no
checking whether pthreads work with -pthread... yes
checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE
checking if more special flags are required for pthreads... no
checking for PTHREAD_PRIO_INHERIT... yes
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating libpcre2-8.pc
config.status: creating libpcre2-16.pc
config.status: creating libpcre2-32.pc
config.status: creating libpcre2-posix.pc
config.status: creating pcre2-config
config.status: creating src/pcre2.h
config.status: creating src/config.h
config.status: executing depfiles commands
config.status: executing libtool commands
config.status: executing script-chmod commands
config.status: executing delete-old-chartables commands

pcre2-10.20 configuration summary:

    Install prefix .................. : /home/kfischer/julia/usr
    C preprocessor .................. : clang -m64  -E
    C compiler ...................... : clang -m64
    Linker .......................... : /usr/bin/ld -m elf_x86_64
    C preprocessor flags ............ :
    C compiler flags ................ : -pthread  -O3 -fvisibility=hidden
    Linker flags .................... :  -Wl,-rpath,/home/kfischer/julia/usr/lib
    Extra libraries ................. :

    Build 8-bit pcre2 library ....... : yes
    Build 16-bit pcre2 library ...... : no
    Build 32-bit pcre2 library ...... : no
    Include debugging code .......... : no
    Enable JIT compiling support .... : yes
    Enable Unicode support .......... : yes
    Newline char/sequence ........... : lf
    \R matches only ANYCRLF ......... : no
    EBCDIC coding ................... : no
    EBCDIC code for NL .............. : n/a
    Rebuild char tables ............. : no
    Use stack recursion ............. : yes
    Internal link size .............. : 2
    Nested parentheses limit ........ : 250
    Match limit ..................... : 10000000
    Match limit recursion ........... : MATCH_LIMIT
    Build shared libs ............... : yes
    Build static libs ............... : yes
    Use JIT in pcre2grep ............ : yes
    Buffer size for pcre2grep ....... : 20480
    Link pcre2grep with libz ........ : no
    Link pcre2grep with libbz2 ...... : no
    Link pcre2test with libedit ..... : no
    Link pcre2test with libreadline . : no
    Valgrind support ................ : no
    Code coverage ................... : no

touch -c pcre2-10.20/config.status
make -C pcre2-10.20
make[1]: Entering directory '/home/kfischer/julia/deps/pcre2-10.20'
 /bin/bash ./config.status
config.status: creating Makefile
config.status: creating libpcre2-8.pc
config.status: creating libpcre2-16.pc
config.status: creating libpcre2-32.pc
config.status: creating libpcre2-posix.pc
config.status: creating pcre2-config
config.status: creating src/pcre2.h
config.status: creating src/config.h
config.status: src/config.h is unchanged
config.status: executing depfiles commands
config.status: executing libtool commands
config.status: executing script-chmod commands
config.status: executing delete-old-chartables commands
rm -f src/pcre2_chartables.c
ln -s /home/kfischer/julia/deps/pcre2-10.20/src/pcre2_chartables.c.dist /home/kfischer/julia/deps/pcre2-10.20/src/pcre2_chartables.c
make  all-am
make[2]: Entering directory '/home/kfischer/julia/deps/pcre2-10.20'
  CC       src/libpcre2_8_la-pcre2_auto_possess.lo
  CC       src/libpcre2_8_la-pcre2_compile.lo
  CC       src/libpcre2_8_la-pcre2_config.lo
  CC       src/libpcre2_8_la-pcre2_context.lo
  CC       src/libpcre2_8_la-pcre2_dfa_match.lo
  CC       src/libpcre2_8_la-pcre2_error.lo
  CC       src/libpcre2_8_la-pcre2_jit_compile.lo
  CC       src/libpcre2_8_la-pcre2_maketables.lo
  CC       src/libpcre2_8_la-pcre2_match.lo
  CC       src/libpcre2_8_la-pcre2_match_data.lo
  CC       src/libpcre2_8_la-pcre2_newline.lo
  CC       src/libpcre2_8_la-pcre2_ord2utf.lo
  CC       src/libpcre2_8_la-pcre2_pattern_info.lo
  CC       src/libpcre2_8_la-pcre2_serialize.lo
  CC       src/libpcre2_8_la-pcre2_string_utils.lo
  CC       src/libpcre2_8_la-pcre2_study.lo
  CC       src/libpcre2_8_la-pcre2_substitute.lo
  CC       src/libpcre2_8_la-pcre2_substring.lo
  CC       src/libpcre2_8_la-pcre2_tables.lo
  CC       src/libpcre2_8_la-pcre2_ucd.lo
  CC       src/libpcre2_8_la-pcre2_valid_utf.lo
  CC       src/libpcre2_8_la-pcre2_xclass.lo
  CC       src/libpcre2_8_la-pcre2_chartables.lo
  CCLD     libpcre2-8.la
  CC       src/libpcre2_posix_la-pcre2posix.lo
  CCLD     libpcre2-posix.la
  CC       src/pcre2grep-pcre2grep.o
  CCLD     pcre2grep
  CC       src/pcre2test-pcre2test.o
  CCLD     pcre2test
  CC       src/pcre2_jit_test-pcre2_jit_test.o
  CCLD     pcre2_jit_test
make[2]: Leaving directory '/home/kfischer/julia/deps/pcre2-10.20'
make[1]: Leaving directory '/home/kfischer/julia/deps/pcre2-10.20'
touch -c pcre2-10.20/.libs/libpcre2-8.so
make -C pcre2-10.20 check -j1
make[1]: Entering directory '/home/kfischer/julia/deps/pcre2-10.20'
make  check-am
make[2]: Entering directory '/home/kfischer/julia/deps/pcre2-10.20'
make
make[3]: Entering directory '/home/kfischer/julia/deps/pcre2-10.20'
make  all-am
make[4]: Entering directory '/home/kfischer/julia/deps/pcre2-10.20'
make[4]: Leaving directory '/home/kfischer/julia/deps/pcre2-10.20'
make[3]: Leaving directory '/home/kfischer/julia/deps/pcre2-10.20'
make  check-TESTS
make[3]: Entering directory '/home/kfischer/julia/deps/pcre2-10.20'
make[4]: Entering directory '/home/kfischer/julia/deps/pcre2-10.20'
./test-driver: line 107: 59457 Segmentation fault      "$@" > $log_file 2>&1
FAIL: pcre2_jit_test

[tkelman]

wtf!? linux? what distro, compiler versions, etc?

[Keno]

kfischer@julia:~/julia$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=15.04
DISTRIB_CODENAME=vivid
DISTRIB_DESCRIPTION="Ubuntu 15.04"
kfischer@julia:~/julia$ cat Make.user
LLVM_VER=svn
#LLVM_DEBUG=1
LLVM_ASSERTIONS=1
BUILD_LLVM_CLANG=1
BUILD_LLDB=1
LLDB_VER=master
USE_LLVM_SHLIB=1
LLDB_DISABLE_PYTHON=1
CXX=clang++
CC=clang
kfischer@julia:~/julia$ clang --version
Ubuntu clang version 3.6.0-2ubuntu1 (tags/RELEASE_360/final) (based on LLVM 3.6.0)
Target: x86_64-pc-linux-gnu
Thread model: posix

[Keno]

This is julia.mit.edu, so you have ssh.

[tkelman]

I can't reproduce, on the same machine with the same Make.user. What gives?

https://gist.github.com/tkelman/0d3daf2ab98c6b4e6242

[Keno]

Curious, I deleted pcre and tried again, and now it passes (though I did the exact same thing twice before filing this issues). Time of Day? Cosmic Rays? Thanks for trying it yourself.

[Keno]

Though curiously the

rm -f src/pcre2_chartables.c
ln -s /home/kfischer/julia/deps/pcre2-10.20/src/pcre2_chartables.c.dist /home/kfischer/julia/deps/pcre2-10.20/src/pcre2_chartables.c

line is no longer in the output

[Keno]

Which is just an artifact of running in a different directory, so I'm left as ❔

[rShekhtman]

I have the exact same error as @Keno . I have tried removing deps/pcre2-10.20 , and recompiling, but this did not help. Is there anything else I should try?

[tkelman]

Try commenting out

julia/deps/Makefile

Line 737 in b5fed62

PCRE_CFLAGS = -O3

or changing it to -O2 or -O1

[rShekhtman]

I have just tried your suggestions. Unfortunately, they did not help me. I am always getting the following error:

./test-driver: line 107: 16399 Segmentation fault      (core dumped) "$@" > $log_file 2>&1
FAIL: pcre2_jit_test

The following is in the pcre2_jit_test.log file:

Running JIT regression tests
  target CPU of SLJIT compiler: x86 64bit (little endian + unaligned)
  in  8 bit mode with UTF-8  enabled:
............................................................
............................................................
............................................................
............................................................
............................................................
............................................................
.............................
8 bit: Cannot compile pattern "a(?=(?C)\B(?C`x`))b": 139
8 bit: Test should match: [390] 'a(?=(?C)\B(?C`x`))b' @ 'ab'

8 bit: Cannot compile pattern "a(?!(?C)\B(?C`x`))bb|ab": 139
8 bit: Test should match: [391] 'a(?!(?C)\B(?C`x`))bb|ab' @ 'abb'

8 bit: Cannot compile pattern "a(?=\b|(?C)\B(?C`x`))b": 139
8 bit: Test should match: [392] 'a(?=\b|(?C)\B(?C`x`))b' @ 'ab'

8 bit: Cannot compile pattern "a(?!\b|(?C)\B(?C`x`))bb|ab": 139
8 bit: Test should match: [393] 'a(?!\b|(?C)\B(?C`x`))bb|ab' @ 'abb'

8 bit: Cannot compile pattern "c(?(?=(?C)\B(?C`x`))ab|a)": 139
8 bit: Test should match: [394] 'c(?(?=(?C)\B(?C`x`))ab|a)' @ 'cab'

8 bit: Cannot compile pattern "c(?(?!(?C)\B(?C`x`))ab|a)": 139
8 bit: Test should match: [395] 'c(?(?!(?C)\B(?C`x`))ab|a)' @ 'cab'

8 bit: Cannot compile pattern "c(?(?=\b|(?C)\B(?C`x`))ab|a)": 139
8 bit: Test should match: [396] 'c(?(?=\b|(?C)\B(?C`x`))ab|a)' @ 'cab'

8 bit: Cannot compile pattern "c(?(?!\b|(?C)\B(?C`x`))ab|a)": 139
8 bit: Test should match: [397] 'c(?(?!\b|(?C)\B(?C`x`))ab|a)' @ 'cab'
........................................................FAIL pcre2_jit_test (exit status: 139)

[tkelman]

Just checking, are you doing make -C deps distclean-pcre (or equivalent manual deletion) between each change?

[rShekhtman]

No, I wasn't doing that. I was only removing the pcre2 directory. I did distclean-pcre, and everything compiled successfully. Thanks for the help.

[timholy]

I ran into this too and the make -C deps distclean-pcre fixed it. Ideally we'd set up some makefile thingy (witness my deep understanding of build processes!) that executes this when pcre versions change.

[yuyichao]

Are they getting extracted (or being built) in the same directory (i.e. non-versioned)?

[tkelman]

For the git-externals ones yes (though there's a makefile rule with a timestamp dependency on libuv.version et al), for the tarball ones it should have a version number in the name - but the dependency doesn't get rebuilt usually unless you do make cleanall.

[dbrowne]

make cleanall resolved the problem for me.

[vtjnash]

very strange. deleting the tar.bz2 file seemed to resolve it for me also, whereas just deleting the folder was not enough -- but they are the the same bits.

[tkelman]

one of the touches that happens on extraction maybe?

[JeffBezanson]

Just ran into this for the first time.

[JeffBezanson]

Tried several cycles of removing everything pcre from deps and rebuilding, to no avail. Removing --enable-jit seemed to cause the build to hang while compiling the pcre2 test program. Tried downgrading to PCRE 10.10 and that fixed it.

[mbaz]

I just ran into this issue when building v0.4.5 on three out of four Ubuntu 14.04 machines. 'make -C deps distclean-pcre' fixed it. One of the machines didn't need it; I've no idea why, since they're identical.