Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

grpc compilation error #68

Closed
mereacre opened this issue Nov 19, 2021 · 9 comments · Fixed by #69
Closed

grpc compilation error #68

mereacre opened this issue Nov 19, 2021 · 9 comments · Fixed by #69
Assignees

Comments

@mereacre
Copy link
Contributor

I get this when compiling:
[ 0%] Generating reverse_access.pb.cc, reverse_access.pb.h, reverse_access.grpc.pb.cc, reverse_access.grpc.pb.h
Scanning dependencies of target reverse_grpc_proto
[ 1%] Building CXX object CMakeFiles/reverse_grpc_proto.dir/reverse_access.grpc.pb.cc.o
In file included from /usr/include/google/protobuf/arenastring.h:38,
from /home/alex/Projects/EDGESec/build/reverse_access.pb.h:24,
from /home/alex/Projects/EDGESec/build/reverse_access.grpc.pb.cc:5:
/usr/local/include/google/protobuf/stubs/fastmem.h:54:10: fatal error: google/protobuf/port_def.inc: No such file or directory
54 | #include <google/protobuf/port_def.inc>
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [CMakeFiles/reverse_grpc_proto.dir/build.make:78: CMakeFiles/reverse_grpc_proto.dir/reverse_access.grpc.pb.cc.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:823: CMakeFiles/reverse_grpc_proto.dir/all] Error 2
make: *** [Makefile:163: all] Error 2

$ protoc --version
libprotoc 3.6.1

@mereacre
Copy link
Contributor Author

-- Found GRPC: /usr/local/include (found version "1.31.1")
-- Found Protobuf: /usr/lib/x86_64-linux-gnu/libprotobuf.so;-lpthread (found version "3.6.1")

@mereacre
Copy link
Contributor Author

It doesn't work with both pre compiled and with the compilation on.

The problem is the missing file google/protobuf/port_def.inc

@mereacre
Copy link
Contributor Author

Correction: It doesn't work with libproto-dev deb installed. It does work with grpc source compilation.

@aloisklink
Copy link
Contributor

My gut feeling is that your version of Protobuf was installed via apt, while your version of GRPC was installed from source, and is therefore newer than Protobuf.

I'd recommend deleting GRPC and just using apt's GRPC

You could try making a PR (or pushing to main) and letting the GitHub Action CI/CD try compiling it.

Or use pbuilder. pbuilder starts from a blank image every time you build a .deb, so it's super slow (no-caching), but it does mean that the environment is clean each time.

@aloisklink
Copy link
Contributor

Hey Alex, I just tried checking out main (commit 3ef2888), deleted build, and rebuilt the entire project and it worked fine.

All tests ran too.

These are the commands I ran:

git switch main
git pull
rm -r build/
# build using multiprocesses
cmake -B build/ -S . -DLIB_MAKEFLAGS="--jobs=$(nproc)"
cmake --build build/ -j4
cmake --build build/ --target test -j4

I'm fairly certain the issue is that you're mixing up having libproto installed from deb, and GRPC installed from src.

Either you should have both protobuf and GRPC from src, or both protobuf and GRPC from apt.

Is there any chance you can delete the build directory, run the giant sudo apt install blah blah blah in the README and recompile?

@mereacre
Copy link
Contributor Author

I did remove the build and run again and it work. But it was after I installed the GRPC from apt. So, I'm not sure that it is still from compiling from source. But you might be right there are problem with grpc and protobuf when it is compiled and installed from apt.

@mereacre
Copy link
Contributor Author

I will keep the issue open for now while I check the problem.

@aloisklink
Copy link
Contributor

Yah, to be honest, if you are building GRPC from source, it should use automatically build protobuf from source too (or at least warn you). I can have a look into it and add a warning.

@aloisklink
Copy link
Contributor

aloisklink commented Nov 22, 2021

Hmmmm, that's super bizarre.

I just tried building GRPC from source (with both protobuf and GRPC installed via apt as well), e.g:

git switch main
git pull
rm -r build/
# build using multiprocesses
cmake -B build/ -S . -DLIB_MAKEFLAGS="--jobs=$(nproc)" -DBUILD_GRPC_LIB=ON
cmake --build build/ -j4
cmake --build build/ --target test -j4

It took forever to download and compile, but it worked! If you build GRPC from source, by default it builds it's own Protobuf (unless you define gRPC_PROTOBUF_PROVIDER and set it to something else).

So I'm not really sure how you got a broken config 🤷

aloisklink added a commit that referenced this issue Nov 30, 2021
Compiling the `.debs` with continuous integration does not like building GRPC from source.

@mereacre, any chance you can try running the following commands on your PC?
(you can just copy-and-paste them into a terminal and wait 30 minutes in the background)

I just want to see if it will work fine to use GRPC from `apt` on a fresh folder.

First, install the dependencies:

```bash
sudo apt update
build_dependencies=(
    cmake # build-tool
    git # required to download dependencies
    ca-certificates # required for git+https downloads
    doxygen texinfo graphviz # documentation
    build-essential # C and C++ compilers
    libnl-genl-3-dev libnl-route-3-dev # netlink dependencies
    automake # required by libmicrohttpd for some reason?
    autopoint gettext # required by libuuid
    autoconf # required by compile_sqlite.sh
    libtool-bin # required by autoconf somewhere
    pkg-config # seems to be required by nDPI
    libjson-c-dev # mystery requirement
    flex bison # required by pcap
    libgnutls28-dev # required by libmicrohttpd
    libssl-dev # required by hostapd only. GRPC uses own version, and we compile OpenSSL 3 for EDGESec
    protobuf-compiler-grpc libprotobuf-dev libgrpc++-dev # GRPC, can be removed if -DBUILD_GRPC_LIB=ON
    libcmocka-dev # cmocka, can be removed if -DBUILD_CMOCKA_LIB=ON
    libmnl-dev # libmnl, can be removed if -DBUILD_LIBMNL_LIB=ON
)
runtime_dependencies=(
    dnsmasq
    jq # required by predictable wifi name script
)
sudo apt install -y "${build_dependencies[@]}" "${runtime_dependencies[@]}"
```

Download, build, and compile src:

```bash
cd /tmp
git clone [email protected]:nqminds/EDGESec.git
cd /tmp/EDGESec
git switch set-grpc-to-use-apt
cmake -B build/ -S . -DLIB_MAKEFLAGS="--jobs=$(nproc)"
cmake --build build/ -j4
cmake --build build/ --target test -j4
```

Closes #68 if it works on your PC.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants