Document how to cross-compile

[alexlarsson]

From @hadess on May 5, 2016 17:25

When we have, in the near future, support for ARM in the Freedesktop and GNOME SDK, it would be good to document how to cross compile for one to the other, if only x86-64 to i686 to start with, and x86-64 to ARM afterwards.

Copied from original issue: alexlarsson/xdg-app#155

[hadess]

http:https://wiki.qemu.org/download/qemu-doc.html#QEMU-User-space-emulator ?

[aleixpol]

What kind of documentation do you want? Currently it's just about passing --arch and hoping binfmt is properly configured, correct?

[hadess]

I'd expect instructions to include:

• Installing the target runtime and SDK
• Verifying that binfmt is correctly configured (launching a flatpak with --arch, and expecting bash to run properly)
• Passing --arch to flatpak-builder, or setting it in the json file if the binaries are targeted only at this specific architecture (eg. not compiling from source)

[aleixpol]

If you tell me where you want this documentation and an example, I can write something.

[bochecha]

Having just gone through that, here's what I ended up doing:

1. install the qemu support

$ sudo dnf install qemu-system-arm qemu-user-static
$ sudo systemctl restart systemd-binfmt.service

2. install the runtime/sdk for the target architecture:

$ flatpak install --user gnome org.freedesktop.Sdk/${ARCH}/1.6 org.freedesktop.Platform/${ARCH}/1.6

3. build:

$ flatpak-builder --arch=${ARCH} [OPTIONS] DIRECTORY MANIFEST

The thing which caused me problems is that I had installed the qemu-user-binfmt package instead of the qemu-user-static one, and that only led to "No such file or directory" errors.

Of course, the packages might be named differently depending on the distribution.

[fedelibre]

There's a nice guide about cross-compilation here:
https://developer.puri.sm/Librem5/Apps/Packaging_Apps/Building_Flatpaks/Cross-Building.html

I've followed it and I'm now trying to use it to build an aarch64 flatpak using an x86_64 laptop. But I get a weird error when building the first module in the manifest:

⬢[fede@toolbox org.frescobaldi.Frescobaldi]$ flatpak-builder --force-clean --arch=aarch64 --repo=myrepo _aarch64 org.frescobaldi.Frescobaldi.yaml
Emptying app dir '_aarch64'
Downloading sources
Starting build of org.frescobaldi.Frescobaldi
Cache miss, checking out last cache hit
========================================================================
Building module guile in /var/home/fede/src/flathub.org/org.frescobaldi.Frescobaldi/.flatpak-builder/build/guile-2
========================================================================
Applying patch guile-1.8.8-deplibs.patch
patching file guile-1.8.pc.in
patching file guile-config/guile-config.in
patching file libguile/Makefile.in
Hunk #2 succeeded at 2126 (offset -5 lines).
Hunk #3 succeeded at 2148 (offset -5 lines).
Applying patch guile-1.8.8-cve-2016-8605.patch
patching file libguile/filesys.c
Hunk #1 succeeded at 791 (offset -464 lines).
Applying patch guile-1.8.8-configure.patch
patching file configure.in
patching file configure
bwrap: execvp /bin/sh: Exec format error
Error: module guile: Child process exited with code 1

Is this something related to binfmt? (as @aleixpol mentioned above)

I'm on Fedora 34 Silverblue and I'm running flatpak-builder within a fedora 34 container, where I installed qemu-user-static.

[fedelibre]

I've just tried building Lollypop and I get an error while building the first module:

$ flatpak-builder --arch=aarch64 --repo=myrepo _aarch64 org.gnome.Lollypop.json 
[...]
Starting build of org.gnome.Lollypop
========================================================================
Building module python3-beautifulsoup4 in /var/home/fede/src/flathub.org/org.gnome.Lollypop/.flatpak-builder/build/python3-beautifulsoup4-1
========================================================================
Running: pip3 install --exists-action=i --no-index --find-links="file:https://${PWD}" --prefix=${FLATPAK_DEST} "beautifulsoup4"
bwrap: execvp /bin/sh: Exec format error
Error: module python3-beautifulsoup4: Child process exited with code 1

What I'm doing wrong?

[hadess]

bwrap: execvp /bin/sh: Exec format error

Your system can't execute aarch64 executables, from the likes of it. Note that this issue is about documenting cross-compilation, not helping folks out with their own problems. You might want to reach out to your distribution's help forums instead.

[nanonyme]

This has become relevant again now that i386 is EOL and you need to use cross-compile Sdk extensions to build 32bit binaries.

[gasinvein]

@nanonyme This issue was about using qemu-user-static to build native compiler targets on machines with foreign arch CPU (e.g. do aarch64 build with aarch64 SDK on a x86_64 machine). This approach doesn't involve cross-compilers, but emulation. So, to use this approach to build i386 apps, one would need actual i386 SDK.

[tinywrkb]

Here's another idea how to handle this.

Create an SDK extension org.freedesktop.Sdk.Extension.runner-aarch64

build.sh

#!/bin/bash

_WRKDIR="$PWD"
_BUILDDIR="$_WRKDIR"/build
_APPNAME=org.freedesktop.Sdk.Extension.runner-aarch64
_SDK=org.freedesktop.Sdk
_RUNTIME=org.freedesktop.Sdk
_BRANCH=21.08
_REPO="$_WRKDIR"/flatpak-repo
_QEMU_DOWNLOAD="https://github.com/multiarch/qemu-user-static/releases/download/v6.1.0-8/qemu-aarch64-static.tar.gz"

flatpak build-init --arch=aarch64 --type=extension --extension-tag=org.freedesktop.Sdk.Extension "$_BUILDDIR" $_APPNAME $_SDK $_RUNTIME $_BRANCH

curl -L "$_QEMU_DOWNLOAD" | bsdtar -Oxf - | install -Dm755 /dev/stdin "$_BUILDDIR"/files/bin/qemu-aarch64-static

flatpak build-finish --arch=aarch64 --type=extension --extension-tag=org.freedesktop.Sdk.Extension "$_BUILDDIR"

flatpak build-export $_REPO "$_BUILDDIR" $_BRANCH

Install the extension.

$ flatpak install --user --arch=aarch64 flatpak-repo org.freedesktop.Sdk.Extension.runner-aarch64//21.08

Update binfmt_misc.

$ sudo install -dm755 /run/binfmt.d
$ echo ':qemu-aarch64:M::\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\xb7\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/lib/sdk/runner-aarch64/bin/qemu-aarch64-static:OC' | sudo tee /run/binfmt.d/qemu-user.conf
$ sudo systemctl restart systemd-binfmt.service

Build your app with Flatpak Builder's --arch=aarch64 option.

[talregev]

Having just gone through that, here's what I ended up doing:

1. install the qemu support

$ sudo dnf install qemu-system-arm qemu-user-static
$ sudo systemctl restart systemd-binfmt.service

2. install the runtime/sdk for the target architecture:

$ flatpak install --user gnome org.freedesktop.Sdk/${ARCH}/1.6 org.freedesktop.Platform/${ARCH}/1.6

3. build:

$ flatpak-builder --arch=${ARCH} [OPTIONS] DIRECTORY MANIFEST

The thing which caused me problems is that I had installed the qemu-user-binfmt package instead of the qemu-user-static one, and that only led to "No such file or directory" errors.

Of course, the packages might be named differently depending on the distribution.

@bochecha Thank you for writing this! Take me long time to understand how to build flatpak for arm.
This is what I really needed.
@aleixpol Why this is not documented? For arch / arm explain in the official docs?
Please add this to the docs:
https://docs.flatpak.org/

[mbridon]

Hi, I used to be called @bochecha but due to $reasons I had to create this new account and abandon that old one because the Github support couldn't just give me back my old account...

You're welcome, I'm glad that documentation I wrote a long time ago could still be useful after all these years 😄

I welcome the idea of moving it to the Flatpak docs 👍

[talregev]

@mbridon Thank you again!
I have an issue, but I think it more global, if you can confirm.
I am no longer able can install these flatpack from arm / aarch64:

• org.gnome.Sdk
• org.gnome.Platform
• org.freedesktop.Platform
• org.freedesktop.Sdk

Yesterday I was able to download and install using flatpak, but today no. Can you verify it from your side?

[mbridon]

I just installed runtime/org.gnome.Sdk/arm/3.36 on my x86_64 machine 🤷

it only complained about this:

Info: org.gnome.Sdk//3.36 is end-of-life, with reason:
   The GNOME 3.36 runtime is no longer supported as of February 13, 2021. Please ask your application developer to migrate to a supported platform.
Info: org.freedesktop.Platform.GL.default//19.08 is end-of-life, with reason:
   The Freedesktop SDK 19.08 runtime is no longer supported as of September 1, 2021. Please ask your application developer to migrate to a supported version

[talregev]

I try to do the same on my local and the gitgub action (CI), both work for me on Saturday, but yesterday and today not:

sudo apt update 
sudo apt install -y flatpak flatpak-builder qemu-system-arm qemu-user-static
sudo flatpak remote-add --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo
sudo flatpak install -y flathub org.gnome.Sdk/${{ matrix.arch }}/42 flathub org.gnome.Platform/${{ matrix.arch }}/42

The matrix.arch are:

      matrix:
        include:
        - type: x64
          arch: x86_64
        - type: arm64
          arch: aarch64

It not found the aarch64.

Looking for matches…
error: Nothing matches org.gnome.Sdk in remote flathub
Error: Process completed with exit code 1.

It it succeeded on Saturday in both (local computer and github action), I think it not a problem on my side. and If it is, I am not sure what I am doing wrong. Thank you for your response.

[talregev]

Can you try with aarch64?

[mbridon]

Same result as arm 🤷

[talregev]

Same result as arm shrug

Thank you for trying!

[talregev]

@mbridon
I understand the issue!
#5080