[GLUTEN-3806][VL] Add Arm64 libhdfs3 support for velox build #6190
Conversation
Change-Id: Ifd356fd6c60fecb4f928b6cd363b74e1e723f945
ep/build-velox/src/get_velox.sh
Outdated
| @@ -137,7 +137,7 @@ function process_setup_centos8 { | |||
| sed -i '/^ run_and_time install_fbthrift/a \ run_and_time install_openssl' scripts/setup-centos8.sh | |||
|
|
|||
| if [ $ENABLE_HDFS == "ON" ]; then | |||
| sed -i '/^function install_gflags.*/i function install_libhdfs3 {\n cd "\${DEPENDENCY_DIR}"\n github_checkout oap-project/libhdfs3 master\n cmake_install\n}\n' scripts/setup-centos8.sh | |||
| sed -i '/^function install_gflags.*/i function install_libhdfs3 {\n cd "\${DEPENDENCY_DIR}"\n github_checkout oap-project/libhdfs3 master\n git apply ../../../src/modify_libhdfs3.patch\n cd ..\n cmake_install\n}\n' scripts/setup-centos8.sh | |||
There was a problem hiding this comment.
can we use CURRENT_DIR ?
There was a problem hiding this comment.
Yes, it seems CURRENT_DIR is deprecated.
Removed it. Thanks.
There was a problem hiding this comment.
The libhdfs codebase is maintained by oap self. I'm wondering if we can push to libhdfs repo directly.
There was a problem hiding this comment.
Hi @ulysses-you Thanks for your comments.
It makes sense to push it to libhdfs repo directly.
I've submitted the related PR in oap-project/libhdfs3.
However, it appears that oap-project/libhdfs3 is an inactive project. I'm unsure about the time it will take for the PR process to conclude.
I've added my comments inline. I'll continue to track this issue and will remove this libhdfs patch as soon as oap-project/libhdfs3 accepts the related PR.
Would it be possible to accept this PR in the meantime?
Thank you. :-)
Change-Id: I069f7febe92a342a0bc2d843093d1ed7cd1e6226 Signed-off-by: Yuqi Gu <[email protected]>
What changes were proposed in this pull request?
SSE4.2is always enabled in Project oap-project/libhdfs3 Options.cmake.When building Gluten-Velox with enabling HDFS(
--enable_hdfs=ON) on Arm64 Ubuntu, build issues occurred:How was this patch tested?
Build Gluten-Velox with --enable_hdfs=ON
./dev/builddeps-veloxbe.sh --build_tests=ON --build_benchmarks=ON --enable_hdfs=ON
libhdfs.so could be properly installed.