Broken build using v0.9.0 release tar

[Edenhofer]

BCC can not be build on an Arch Linux 4.20.14 system using the v0.9.0 release tar from GitHub. However, it can be build using the git repository and checking out the v0.9.0 tag. I successfully reproduced the error on another Arch machine with a different configuration. Hence, I do not think the problem is device specific. Furthermore, the previous version v0.8.0 builds fine.

I used the following commands for building

mkdir build; cd build
cmake -DCMAKE_INSTALL_PREFIX=/usr ..
make

and got the following error message

bcc-0.9.0/src/cc/libbpf.c:54:10: fatal error: libbpf/src/bpf.h: No such file or directory
 #include "libbpf/src/bpf.h"
          ^~~~~~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [src/cc/CMakeFiles/bpf-shared.dir/build.make:63: src/cc/CMakeFiles/bpf-shared.dir/libbpf.c.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:163: src/cc/CMakeFiles/bpf-shared.dir/all] Error 2
make: *** [Makefile:141: all] Error 2

To be honest I am a little bit confused as to what might be the cause of the error. It might very well just me doing something wrong. Any help would be greatly appreciated.

[r4f4]

I've hit the same problem trying to build from the release tar in Fedora.

[r4f4]

[bcc-0.9.0]$: ls src/cc/
api              bpf_module.cc                     libbpf.h
bcc_btf.cc       bpf_module.h                      link_all.cc
bcc_btf.h        bpf_module_rw_engine.cc           ns_guard.cc
bcc_common.cc    bpf_module_rw_engine_disabled.cc  ns_guard.h
bcc_common.h     clang                             perf_reader.c
bcc_debug.cc     CMakeLists.txt                    perf_reader.h
bcc_debug.h      common.cc                         README
bcc_elf.c        common.h                          setns.h
bcc_elf.h        compat                            shared_table.cc
bcc_exception.h  export                            syms.h
bcc_perf_map.c   exported_files.cc                 table_desc.h
bcc_perf_map.h   exported_files.h                  table_storage.cc
bcc_proc.c       file_desc.h                       table_storage.h
bcc_proc.h       frontends                         table_storage_impl.h
bcc_syms.cc      json_map_decl_visitor.cc          usdt
bcc_syms.h       libbcc.pc.in                      usdt.h
bcc_usdt.h       libbpf                            vendor
bpffs_table.cc   libbpf.c
[bcc-0.9.0]$: ls src/cc/libbpf/
[bcc-0.9.0]$: 

So the libbpf is empty. Are we supposed to git submodule init it?

[yonghong-song]

We have the following at bcc/CMakeLists.txt:

# populate submodules (libbpf)
if(NOT EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/src/cc/libbpf/src)
  execute_process(COMMAND git submodule update --init --recursive
                  WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR})
endif()

Could you help check why git submodule not running?

[Edenhofer]

The snippet from bcc/CMakeLists.txt does not solve the problem if it is shipped as part of the tar file, as no information about the git submodules is preserved. Hence, the build is implicitly relying on the git-repository which to me seems kind of odd.

@yonghong-song Thanks a bunch for providing the missing piece to the puzzle!

[Edenhofer]

It seems like the snippet is aiming at satisfying dependencies using git. Please don't do this. It makes it a lot harder to package bcc nicely.

[yonghong-song]

After checkout, we should do git submodule update --init --recursive before generating the tar file. This way, the tar file is self contained. If you have cycles, could you take a look? Maybe some scripts under bcc/scripts needs to be updated?

[Edenhofer]

The tar files are created by GitHub. The files are one-to-one representations of the files in the repository. AFAIK they can not be changed.

[yonghong-song]

Yes, looks like this is a known issue many people experienced. https://github.community/t5/How-to-use-Git-and-GitHub/Create-Release-that-contains-submodule/td-p/12073.

[Edenhofer]

IMHO the submodule should not be part of the tar file. Either the submodule is as closely coupled to the project as the project is with the rest of its internal code, in which case it should actually belong to the same project or it is not, in which case it should be treated as just another dependency.

[leahneukirchen]

If it's "just another dependency" then libbpf also should have proper releases.

[Edenhofer]

Right, that is what I would hope for if it is categorized as a dependency.

[yonghong-song]

Good point, I will propose and tag libbpf properly. How about tag name "bcc_release_<release_num>"?

[Edenhofer]

IMHO the versions should not be coupled directly via a prefix in the tag if libbpf is really decoupled from bcc and e.g. bcc v0.9.0 works with libbpf v0.9.0 and v0.8.0. If I were to choose, I would probably just tag libbpf using the same schema as bcc but without any additional prefix in front of the tag.

[yonghong-song]

We have some discussion on this issue during today's iovisor bi-weekly meeting. One idea is since bcc knows the exact commit hash for the submodule, maybe it can record this hash in the repo, so this hash also available in the github generated tar file. When users run cmake, the build system will first try to do git submodule init. If that is not successful, maybe find this hash, do a checkout with this hash at the specific location. This should solve this issue and it is transparent to the user as well.

[Edenhofer]

I would highly welcome proper releases for libbpf if it shall be treated as just an other dependencies. Downloading libraries during configuration of the software seems odd to me and messes with build tools like Arch's makepkg which expects all files to be downloaded prior to configuring and building the software.

Plus this would add git to the list of dependencies just to download a library.

[r4f4]

Is there any resolution to this issue?

[rickysarraf]

This has stuck Debian too. bcc right now is removed from the Debian Testing repository because the older version, 0.8.0, had dependencies on older deprecated packages.

Have others on this bug report found any workaround for this bug ? It has been so long since it was reported.
At least on Debian, the build infrastructure disables network access, so fetching something during configure/build time is not available, by default

[Edenhofer]

For now I manually check which libbpf commit is referenced in the release, downloaded it and move it to the right location. See the Arch Linux PKGBUILD. It should be pretty self-explanatory since it is plain shell code.

[r4f4]

I'm generating my own tarball from the repo and using that for the build.

git clone -b "v${version}" --single-branch --depth 1 ${url} bcc-${version}
pushd bcc-${version}
git submodule update --init
popd
tar zcvf bcc-${version}.tar.gz bcc-${version}/

[rickysarraf]

Thanks @Edenhofer @r4f4

So hard to solve upstream...

[rickysarraf]

And we also have the packages named and provided through the linux kernel.

libbpf-dev - eBPF helper library (development files)
libbpf0 - eBPF helper library (shared library)
libbpf4.19 - eBPF helper library (shared library)

[yonghong-song]

I just manually update the bcc source code with proper submodules populated, see https://github.com/iovisor/bcc/releases

bcc-src-with-submodule.tar.gz 2.64 MB
bcc-src-with-submodule.zip 3.18 MB
Source code (zip)
Source code (tar.gz) 

do you think this will partially solve your problem?

[rickysarraf]

@yonghong-song Whichever way you prefer, I would request this be documented proper. This bug report was created for the same lack of information.

For now, in Debian, I chose to keep it as multiple sources. I just, yesterday, pushed the 0.11.0 release to Debian and it should now soon show up in the archives, followed by the many Debian derivatives out there.

Debian packaging tools do have intelligence in place to deal with a package which can have multiple upstream source tarballs.

[yonghong-song]

@rickysarraf Make sense. I will add some documentation later.

[samliddicott]

I was able to resolve this for bitbake builds with:

SRC_URI += "git:https://github.com/libbpf/libbpf.git;protocol=https;name=libbpf;destsuffix=git/src/cc/libbpf;branch=master"
SRCREV_libbpf = "c5389a965bc3f19e07b1ee161092fc227e364e94"

So cmake no longer runs git submodule to fetch libbpf sources during build as the source is already present after do_unpack