Support meson build system

[Jakuje]

Problem Description

As proposed by @jpakkane in #2567 (comment) the autotools build system is outdated, inflexible and can cause a barrier for new users interested in contributing to opensc.

Right now, we have autotools and some windows Makefile.mak which are used to build the windows binaries on appveyor.

Proposed Resolution

Implement meson build system to simplify building opensc.

We would probably have to keep both of them for some transition period.

[metsma]

I vote for CMake.
Better support with IDEs.

[frankmorgner]

Build systems come and go. When I have looked deeper into this, scons was the hot stuff (and just like meson, it was written in python). However, for most of my projects I still stuck to autotools, because the tools are easily available everywhere except on Windows. So when switching to some new build system we need to make sure that it still exists in 10 years from now.

The original request was to minimize friction when developing/changing OpenSC (with meson being capable of offering that). However, having looked deeply into multiple build systems in the past, my summary is that if you want to do complicated things, then the build configuration looks complicated as well. And although I'm not familiar with meson, I know that modern cmake can do this fairly well. It is one tool, that can configure, build and install your software, it can generate a build configuration for your IDE/build tool and there are tons of packages that can be used without additional installation (I like this package manager, for example).

So all in all I'd also opt for cmake, because it is better established than all the new kids in town and it equally reduces developer friction.

[mouse07410]

IMHO: no scons, no conan, no meson, no yarn, no .

[eli-schwartz]

However, for most of my projects I still stuck to autotools, because the tools are easily available everywhere except on Windows. So when switching to some new build system we need to make sure that it still exists in 10 years from now.

Scons, meson, and cmake are all at least a decade old, there's quite some staying power with that. But scons isn't "like meson, written in python". The language implementation of a program is not the distinction of that program.

Scons is literally a library toolkit for writing your own build system in python, while Meson is a DSL that isn't python.

Not relying on a programming language to write build definitions is a good thing. It means that people can more easily recreate that build system themselves, if they need to, to make sure it still exists for their use cases. In fact, there's a c99 implementation of Meson, created by someone who just wanted a version of Meson that is easily bootstrappable on operating systems that don't have python.

As far as whether Meson exists in 10 years, the GNOME ecosystem has made it a goal to use Meson everywhere, and is largely successful in that goal. Mesa, systemd, gstreamer, xorg and wayland, and a variety of Freedesktop software use it. To give some particularly notable names. OpenSC would not be a trend-setter in choosing Meson, or running the risk of betting on some obscure new build system no one uses.

While it's certainly true that "which build system to prefer" can be deeply personal and I hesitate to call anyone "wrong" for preferring cmake because they like it better... I don't think it's fair to claim that cmake is somehow the best choice because only cmake is an established build system. ????

It's certainly worth considering to use a well-established build system called "Meson" if someone else puts in the work to port it, as I believe @jpakkane was intending to do...

It is one tool, that can configure, build and install your software

No it isn't. Just like Meson, cmake is a generator, not a builder. It creates Makefiles or ninja files or Visual Studio / Xcode solutions, and then after running cmake, you run make/ninja/whatever.

Scons actually does use its own internal builder, rather than depending on something like make/ninja, though. (This is often a criticism of scons; it's slow.)

cmake does have cmake --build builddir/, which detects that you generated an e.g. ninja build description and runs ninja on your behalf but maybe runs make or msbuild depending on your chosen generator. (Meson calls this meson compile -C builddir/, same principle.)

it can generate a build configuration for your IDE/build tool

Meson does too, BTW: https://mesonbuild.com/IDE-integration.html (there's a partial list of existing IDE plugins at the bottom). Anything that doesn't have a Meson-specific integration can probably use a clang compilation database, which Meson creates -- for example, CLion can be used that way.

there are tons of packages that can be used without additional installation (I like this package manager, for example).

conan runs before you run cmake, it installs dependencies which cmake then detects. But Meson also detects those same dependencies. This is a good thing, because non-Windows users really don't like it when software tries to build its own vendored dependencies.

Meson has https://mesonbuild.com/Wrap-dependency-system-manual.html which actually makes this work correctly -- you choose how you want to look up dependencies, by default "try to find an existing copy, if not, download and build a vendored copy" and drop a *.ini file into the source tree to register dependencies as possibly-vendored. You don't need to edit the build definitions, even.

[frankmorgner]

I have a little experience with cmake and no experience with meson. But what I can see from a high level perspective is that cmake's community is very active with conference talks and new features/modules and that many other projects are switching to it. cmake offers everything that was requested to reduce developer friction and it seems to be a safe bet.

On the other hand, if I want to use meson, the first thing that https://mesonbuild.com/Getting-meson.html wants me to is to install a specific version of python (note the remarks about "bad" versions of python https://mesonbuild.com/Getting-meson.html#platformspecific-install-quirks). Second, going to https://github.com/mesonbuild/meson/releases for downloads, I notice that msi files are completely missing (although being advertised https://mesonbuild.com/Getting-meson.html#installing-meson-and-ninja-with-the-msi-installer). This are just my first impressions on meson, but this is exactly the kind of distractions, I do not want to have!

But let's not make this a editor buildsystem war. We could collect votes from maintainers and developers to decide this. After all, someone needs to do the initial work and the maintenance...

[jpakkane]

for downloads, I notice that msi files are completely missing (although being advertised

This is due to the fact that at the time the release was made, there was a bug in the installer generator framework we use to build the .msi images. Upstream has since fixed the issue and I have just created and uploaded the package.

[eli-schwartz]

But let's not make this a editor buildsystem war. We could collect votes from maintainers and developers to decide this. After all, someone needs to do the initial work and the maintenance...

I completely, totally, 100% agree!

But, this is a ticket for "someone is willing to contribute Meson, how about it". Which is why I thought it was a bit unusual to see what amounts to "no, even if Meson is contributed I would reject it because I don't trust Meson to exist in the long term. Better use cmake instead".

:)

I don't think this is a fair assessment, and I pointed out a few reasons above why I don't think so.

the first thing that https://mesonbuild.com/Getting-meson.html wants me to is to install a specific version of python (note the remarks about "bad" versions of python

msys2 is a complex framework, that isn't even one environment -- it's 7 environments, depending on which of 6 different native Windows ABI/toolchains you want to use (gcc/clang, msvcrt/ucrt, 32-bit/64-bit/aarch64) and the 7th being the POSIX emulation layer.

Just like there are 7 versions of python, there are also 7 versions of cmake. The important thing is that you should usually not use the one which expects by default to compile for the POSIX emulation layer. That's it.

There are no "bad" versions of python, nor are there "bad" versions of cmake, especially if you don't use the msys2 POSIX emulator software and just use the standard version of python/cmake provided by the authors of python/cmake. 🎉

I hope this puts your mind at ease that offers to contribute Meson meet the minimum threshold of being an option on the ballot for maintainers and developers to vote on. :)

[mouse07410]

I'm not sure how to handle this. On the one hand, if somebody wants to build OpenSC with meson, it's fine with me. On the other hand, my questions are: how would it affect the current build tools, who's going to maintain this meson build if the main contributor wins a lottery or is run over by a bus, etc.?

[frankmorgner]

hi! any progress on this?

[Jakuje]

hi! any progress on this?

I am still interested in having this, but it did not have enough priority so far. Certainly it was useful as many people already provided their inputs to the change proposal. I am not fixed on Meson as I can probably handle the CMake too, but I find it sometimes over-complicated for simple tasks.

[frankmorgner]

Right, cmake looks very confusing sometimes, but although meson may me more structured, someone has to do the initial work. In both cases this is quite a lot of work considering the number of tools we're using and platforms we're supporting.

Why I still want this, is to get rid of the multiple build systems and tools we currently have in use, which is a high barrier for most developers. cmake and meson would both simplify setting up a build environment.

[mouse07410]

Why I still want this, is to get rid of the multiple build systems and tools we currently have in use, which is a high barrier for most developers. cmake and meson would both simplify setting up a build environment.

Could you please explain who ones of the currently used build tools will be replaced by cmake or meson?

And do you mean "replaced" as "won't be needed anymore", or "will be invoked transparently for the user by the new build system, so still have to be installed/configured/present"?

[jpakkane]

I created a preliminary port to Meson, which is here.

• It builds all libs, exes and some of the tests
• The executables run but segfault when you try to do anything except -h
• Only tested on Linux
• There are some issues with the symbols that are loaded dynamically
• Symbol visibility is not supported, the current code uses a linker script and it might make more sense to upgrade to more portable symbol visibility attributes as they allow the compiler to do more optimizations (and Meson has builtin support for them)
• Nothing is installed yet
• The original build system had some defines passed as compiler arguments but I put those in config.h instead for simplicity.

[frankmorgner]

This doesn't build for me. -ldl is missing as well as a lot of fine tuning from configure.ac. This fine tuning is what makes the build files complicated, actually. However, it's a good starter, thank you!

[jpakkane]

That's missing because I did the port on distro that has the new-style glibc which has dl builtin so you no longer need it. That can be fixed by dl_dep = dependency('dl') and then adding that to the targets that require it.

[frankmorgner]

I've added #2754 , which builds on my machine with 22360bf. However, adding an sm option doesn't work as described in the documentation, you maybe want to have a look on that...

[eli-schwartz]

It looks like the problem is that the option you added doesn't conform to the documentation. Options have types, and you added a boolean option, which means get_option('sm') returns a boolean object. But only "feature" objects have the tristate object methods such as "enabled()" and "disable_auto_if()" etc.

The use of .enabled() will convert a tristate object of type "feature" into a simple boolean. Since you already have a boolean, just use that boolean as-is and drop the .enabled().

(The purpose of the special feature object is to implement fiddly options such as autotools style --with-foobar={yes|no|check}, which compare three string values and either look for or skip a PKG_CHECK_MODULES, and either error out if not found or log that it wasn't found and is being ignored. Meson's type validation prevents you from accidentally using one of these special objects in cases where you expected to find a boolean.)

[frankmorgner]

What's the status of this topic? last time I tried, my local meson build worked to some extent, but I did not have the perseverance to fill all the gaps...

[jpakkane]

I have not touched it since last time. The main reason for this being that the errors I was getting were very much domain specific and didn't make much sense to me If there's something build related you want me to check into, I can do that.

[martinpaljak]

The latest and greatest incident with xz should prioritize getting rid of the cruft and providing simplification. In the end it is not about what will be around in 10 years, but keeping the build process "simple" enough to be able to move to something different.

Maybe having a split between "compiling" and "packaging" would also help (must not necessarily be done by same build tooling) and there might be differences in what various stakeholders find as the best solution - ease of setup and repeatability for (new) developers vs ease of integration with existing tooling for inert organizations (like Linux distros)