Basic setup for benchmarks

[franziskuskiefer]

This needs cleanup and better integration. It's only a basic setup to show how it is supposed to work.

#15 #36

[franziskuskiefer]

@duesee can you review this so we can get it in?

[duesee]

Generally this looks good to me!

What I am unsure about: Similar to the case that we have to think about running build --instrument before test --instrument we now introduce colliding workflows. From my understanding, it can happen that a build --benchmarks run overrides the output of a build --tests run, which may lead to confusing errors. Thus, we need to be a bit careful what to run when.

Would it be difficult to output the benchmarks into a separate folder build/Debug/benchmark? If so, I still think it is okay for now as it is more important to get some numbers.

[franziskuskiefer]

Yeah this is certainly not done yet. It's just adding the basic framework.
Let's get this in and address all the issues once the testing is done.

[duesee]

I've rebased on main. Let's get this in 👍

[franziskuskiefer]

I've rebased on main. Let's get this in 👍

thanks, but please stop force pushing on other people's branches