Make dependency on LLVM native target optional (iovisor#2080)

* Make dependency on LLVM native target optional

Adds an option ENABLE_LLVM_NATIVECODEGEN with default value ON.
If set to off the "nativecodegen" llvm will not be enabled, thus
reducing dependencies on needed libraries (reduced text size when
building with statically linked libraries).

Code that uses native target will not be compiled reducing text size.
Currently this affects the rw_engine which needs the native target.

BPF api "rw_engine_enabled" will have default value "true" if
ENABLE_LLVM_NATIVECODEGEN="ON" and "false" if
ENABLE_LLVM_NATIVECODEGEN="OFF"

Not needed for BCC to work. It somehow brought in the interpreter and
executionengine which is needed. Those features are added instead.

* Remove garbage in code making it compile again

* Remove interpreter and executionengine LLVM dependencies

These doesn't seem to be needed on a Ubuntu 18.04 system (although
executionengine is heavily used).

Interpreter was added due to runtime dependency on ARM64. It brings in
a dependency on ffi library.

(.text._ZL10ffiTypeForPN4llvm4TypeE+0x3a): undefined reference to `ffi_type_float'
(.text._ZL10ffiTypeForPN4llvm4TypeE+0x43): undefined reference to `ffi_type_void'
(.text._ZL10ffiTypeForPN4llvm4TypeE+0x53): undefined reference to `ffi_type_pointer'
(.text._ZL10ffiTypeForPN4llvm4TypeE+0x63): undefined reference to `ffi_type_double'
(.text._ZL10ffiTypeForPN4llvm4TypeE+0x78): undefined reference to `ffi_type_sint8'
(.text._ZL10ffiTypeForPN4llvm4TypeE+0x83): undefined reference to `ffi_type_sint16'
(.text._ZL10ffiTypeForPN4llvm4TypeE+0x93): undefined reference to `ffi_type_sint64'
(.text._ZL10ffiTypeForPN4llvm4TypeE+0xb3): undefined reference to `ffi_type_sint32'
/usr/lib/llvm-6.0/lib/libLLVMInterpreter.a

CMakeLists.txt

@@ -16,6 +16,7 @@ include(GNUInstallDirs)
 include(CheckCXXCompilerFlag)
 include(cmake/FindCompilerFlag.cmake)
 
+option(ENABLE_LLVM_NATIVECODEGEN "Enable use of llvm nativecodegen module (needed by rw-engine)" ON)
 option(ENABLE_RTTI "Enable compiling with real time type information" OFF)
 option(ENABLE_LLVM_SHARED "Enable linking LLVM as a shared library" OFF)
 option(ENABLE_CLANG_JIT "Enable Loading BPF through Clang Frontend" ON)

cmake/clang_libs.cmake

@@ -2,7 +2,10 @@ if(ENABLE_LLVM_SHARED)
 set(llvm_libs "LLVM")
 else()
 set(llvm_raw_libs bitwriter bpfcodegen debuginfodwarf irreader linker
- mcjit objcarcopts option passes nativecodegen lto)
+ mcjit objcarcopts option passes lto)
+if(ENABLE_LLVM_NATIVECODEGEN)
+set(llvm_raw_libs ${llvm_raw_libs} nativecodegen)
+endif()
 list(FIND LLVM_AVAILABLE_LIBS "LLVMCoverage" _llvm_coverage)
 if (${_llvm_coverage} GREATER -1)
  list(APPEND llvm_raw_libs coverage)

src/cc/CMakeLists.txt

@@ -34,6 +34,12 @@ if (${LLVM_PACKAGE_VERSION} VERSION_EQUAL 6 OR ${LLVM_PACKAGE_VERSION} VERSION_G
  set(bcc_common_sources ${bcc_common_sources} bcc_debug.cc)
 endif()
 
+if(ENABLE_LLVM_NATIVECODEGEN)
+set(bcc_common_sources ${bcc_common_sources} bpf_module_rw_engine.cc)
+else()
+set(bcc_common_sources ${bcc_common_sources} bpf_module_rw_engine_disabled.cc)
+endif()
+
 set(bcc_table_sources table_storage.cc shared_table.cc bpffs_table.cc json_map_decl_visitor.cc)
 set(bcc_util_sources ns_guard.cc common.cc)
 set(bcc_sym_sources bcc_syms.cc bcc_elf.c bcc_perf_map.c bcc_proc.c)

src/cc/api/BPF.h

@@ -47,7 +47,7 @@ class BPF {
  static const int BPF_MAX_STACK_DEPTH = 127;
 
  explicit BPF(unsigned int flag = 0, TableStorage* ts = nullptr,
- bool rw_engine_enabled = true, const std::string &maps_ns = "")
+ bool rw_engine_enabled = bpf_module_rw_engine_enabled(), const std::string &maps_ns = "")
  : flag_(flag),
  bpf_module_(new BPFModule(flag, ts, rw_engine_enabled, maps_ns)) {}
  StatusTuple init(const std::string& bpf_program,