Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: Build with ubuntu-24.04 #5919

Merged
merged 9 commits into from
Jul 1, 2024

Conversation

ElectricalBoy
Copy link
Contributor

@ElectricalBoy ElectricalBoy commented Jun 29, 2024

This PR adds a GHA workflow that runs on ubuntu-24.04.

Context

Latest Linux distros (e.g., Ubuntu 24.04, Fedora 40) ship with newer versions of WebKit than what is used in the current GHA workflow (libwebkit2gtk-4.0). Making the matter even worse, the newer version (i.e., libwebkit2gtk-4.1) is not available on the ubuntu-20.04 runner image and the older libwebkit2gtk-4.0 is not available on the latest distros.
To mitigate this issue, this PR creates another build using the ubuntu-24.04 image, forcing the newer version of WebKit to be used in the build.

Verification

  • Check build success on ubuntu-24.04 runner
  • Verify the build from ubuntu-24.04 running on Ubuntu 24.04

Attribution / Credit

This PR is inspired from the approach taken by PrusaSlicer v2.8.0. All commits in this PR are my own though.

@ElectricalBoy

This comment was marked as outdated.

@ElectricalBoy
Copy link
Contributor Author

# check which version of libwebkit2gtk is available
apt show --quiet libwebkit2gtk-4.0-dev >/dev/null 2>&1

This part of the build script (added by #5135) is problematic because due to the inexistence of libwebkit2gtk-4.0-dev, apt show quietly exits with code 100, resulting as a build failure.

@ElectricalBoy ElectricalBoy marked this pull request as ready for review June 29, 2024 05:50
@SoftFever
Copy link
Owner

# check which version of libwebkit2gtk is available
apt show --quiet libwebkit2gtk-4.0-dev >/dev/null 2>&1

This part of the build script (added by #5135) is problematic because due to the inexistence of libwebkit2gtk-4.0-dev, apt show quietly exits with code 100, resulting as a build failure.

@buzzhuzz may be able to shed some light here.

@ElectricalBoy
Copy link
Contributor Author

ElectricalBoy commented Jun 29, 2024

# check which version of libwebkit2gtk is available
apt show --quiet libwebkit2gtk-4.0-dev >/dev/null 2>&1

This part of the build script (added by #5135) is problematic because due to the inexistence of libwebkit2gtk-4.0-dev, apt show quietly exits with code 100, resulting as a build failure.

@buzzhuzz may be able to shed some light here.

That's been resolved with d8c88b4 😄

What remains to be fixed, however, is renaming the stuff so that the built AppImage can successfully be published. Ignore whatever I said here. Gotta take a closer look at the log.

@ElectricalBoy
Copy link
Contributor Author

ElectricalBoy commented Jun 29, 2024

So here is a better analysis on https://github.com/SoftFever/OrcaSlicer/actions/runs/9721907623/job/26835686880: AppImage requires FUSE 2, which is not available by default on Ubuntu 22.04 and 24.04 (they both ship with FUSE 3 instead), as a runtime dependency. c2eb21c tries to mitigate this by manually installing FUSE 2 during the workflow, but this means that users may need to install FUSE 2 to launch the built OrcaSlicer binary (or possibly, any AppImage in general). Gotta see how the build goes first though.

@ElectricalBoy

This comment was marked as outdated.

@ElectricalBoy ElectricalBoy marked this pull request as ready for review June 29, 2024 13:25
@ElectricalBoy
Copy link
Contributor Author

Can confirm the build successfully running on Ubuntu 24.04. Hurray!
I cannot say anything about the "users may need to install FUSE 2 separately" problem, but that is outside the scope of this PR and the OrcaSlicer repository after all.

@ElectricalBoy

This comment was marked as outdated.

@cochcoder
Copy link
Contributor

cochcoder commented Jun 30, 2024

Once ubuntu-24.04 comes out of beta and is ubuntu-latest, should we consider changing this over to ubuntu-latest so that it automatically keeps the Appimage in sync with the latest Linux distros? Then ubuntu-20.04 could updated separately to newer versions if needed to support older/slower distros, like Debian. Until everything works with the latest version of course.

cochcoder added a commit to cochcoder/OrcaSlicer that referenced this pull request Jun 30, 2024
@ElectricalBoy

This comment was marked as outdated.

@ElectricalBoy
Copy link
Contributor Author

Once ubuntu-24.04 comes out of beta and is ubuntu-latest, should we consider changing this over to ubuntu-latest so that it automatically keeps the Appimage in sync with the latest Linux distros? Then ubuntu-20.04 could updated separately to newer versions if needed to support older/slower distros, like Debian. Until everything works with the latest version of course.

We cannot really tell a priori what changes will be required to the workflow when ubuntu-26.04 eventually becomes ubuntu-latest after two years from now, so I think it is more appropriate to assess the migration and do it manually.

cochcoder added a commit to cochcoder/OrcaSlicer that referenced this pull request Jul 1, 2024
@cochcoder cochcoder mentioned this pull request Jul 1, 2024
@Ocraftyone
Copy link
Contributor

From some of my previous testing, you wouldn't be able to any binaries built on a new version of Ubuntu on an older version. If i remember, it was glib causing issues because it couldn't find the correct version.

@ElectricalBoy
Copy link
Contributor Author

ElectricalBoy commented Jul 1, 2024

From some of my previous testing, you wouldn't be able to any binaries built on a new version of Ubuntu on an older version. If i remember, it was glib causing issues because it couldn't find the correct version.

Do you mean that binaries built on a newer version of Ubuntu cannot be run on an older version of Ubuntu? You are probably right, but I don't think it is relevant with this PR's goal.

This PR's goal is not to replace the existing workflow that runs on and generates binaries from ubuntu-20.04; what this PR does is adding a new workflow that runs on ubuntu-24.04 to build binaries for newer Linux distros due to the incompatibility as described in the OP.

To reiterate my point, this PR's goal is to publish two Linux binaries; one built on ubuntu-20.04 (i.e., what we already have running) and another built on ubuntu-24.04 (i.e., what this PR is proposing to add).

@Ocraftyone
Copy link
Contributor

From some of my previous testing, you wouldn't be able to any binaries built on a new version of Ubuntu on an older version. If i remember, it was glib causing issues because it couldn't find the correct version.

Do you mean that binaries built on a newer version of Ubuntu cannot be run on an older version of Ubuntu? You are probably right, but I don't think it is relevant with this PR's goal.

This PR's goal is not to replace the existing workflow that runs on and generates binaries from ubuntu-20.04; what this PR does is adding a new workflow that runs on ubuntu-24.04 to build binaries for newer Linux distros due to the incompatibility as described in the OP.

To reiterate my point, this PR's goal is to publish two Linux binaries; one built on ubuntu-20.04 (i.e., what we already have running) and another built on ubuntu-24.04 (i.e., what this PR is proposing to add).

Ah! Makes much more sense! I didn't RTFM :)

@Ocraftyone
Copy link
Contributor

Patch File

diff --git a/.github/workflows/build_deps.yml b/.github/workflows/build_deps.yml
--- .github/workflows/build_deps.yml
+++ .github/workflows/build_deps.yml
@@ -81,30 +81,22 @@
             ./build_release_macos.sh -dpx -a ${{ inputs.arch }} -t 10.15 -1
             brew install zstd
 
 
-      - name: Install Dependencies (Ubuntu 20.04)
-        if: inputs.os == 'ubuntu-20.04'
+      - name: Install Ubuntu Build Dependencies
+        if: inputs.os == 'ubuntu-20.04' || inputs.os == 'ubuntu-24.04'
         working-directory: ${{ github.workspace }}
+        env:
+          apt-cmd: ${{ (inputs.os == 'ubuntu-20.04' && 'apt-fast') || (inputs.os == 'ubuntu-24.04' && 'sudo apt-get') || '' }}
+          webkit-ver: ${{ (inputs.os == 'ubuntu-20.04' && '4.0') || (inputs.os == 'ubuntu-24.04' && '4.1') || '' }}
         run: |
-            apt-fast update
-            apt-fast install -y cmake git g++ build-essential libgl1-mesa-dev m4 \
+            ${{ env.apt-cmd }} update
+            ${{ env.apt-cmd }} install -y cmake git g++ build-essential libgl1-mesa-dev m4 \
             libwayland-dev libxkbcommon-dev wayland-protocols extra-cmake-modules pkgconf \
-            libglu1-mesa-dev libcairo2-dev libgtk-3-dev libsoup2.4-dev libwebkit2gtk-4.0-dev \
+            libglu1-mesa-dev libcairo2-dev libgtk-3-dev libsoup2.4-dev libwebkit2gtk-${{ env.webkit-ver }}-dev \
             libgstreamer1.0-dev libgstreamer-plugins-good1.0-dev libgstreamer-plugins-base1.0-dev \
             gstreamer1.0-plugins-bad libosmesa6-dev wget sudo autoconf curl libunwind-dev texinfo
 
-      - name: Install Dependencies (Ubuntu 24.04)
-        if: inputs.os == 'ubuntu-24.04'
-        working-directory: ${{ github.workspace }}
-        run: |
-            sudo apt-get update
-            sudo apt-get install -y cmake git g++ build-essential libgl1-mesa-dev m4 \
-            libwayland-dev libxkbcommon-dev wayland-protocols extra-cmake-modules pkgconf \
-            libglu1-mesa-dev libcairo2-dev libgtk-3-dev libsoup2.4-dev libwebkit2gtk-4.1-dev \
-            libgstreamer1.0-dev libgstreamer-plugins-good1.0-dev libgstreamer-plugins-base1.0-dev \
-            gstreamer1.0-plugins-bad libosmesa6-dev wget sudo autoconf curl libunwind-dev texinfo
-
       - name: Build on Ubuntu
         if: inputs.os == 'ubuntu-20.04' || inputs.os == 'ubuntu-24.04'
         working-directory: ${{ github.workspace }}
         run: |
@@ -131,22 +123,17 @@
         with:
           name: OrcaSlicer_dep_win64_${{ env.date }}
           path: ${{ github.workspace }}/deps/build/OrcaSlicer_dep*.zip
 
-      - name: Upload Ubuntu 20.04 artifacts
-        if: ${{ ! env.ACT && inputs.os == 'ubuntu-20.04' }}
+      - name: Upload Ubuntu artifacts
+        if: ${{ ! env.ACT && inputs.os == 'ubuntu-20.04' || inputs.os == 'ubuntu-24.04' }}
+        env:
+          ubuntu-ver: ${{ (inputs.os == 'ubuntu-20.04' && '2004') || (inputs.os == 'ubuntu-24.04' && '2404') || '' }}
         uses: actions/upload-artifact@v4
         with:
-          name: OrcaSlicer_dep_ubuntu_${{ env.date }}
+          name: OrcaSlicer_dep_ubuntu_${{ env.ubuntu-ver }}_${{ env.date }}
           path: ${{ github.workspace }}/deps/build/OrcaSlicer_dep_ubuntu_*.tar.gz
 
-      - name: Upload Ubuntu 24.04 artifacts
-        if: ${{ ! env.ACT && inputs.os == 'ubuntu-24.04' }}
-        uses: actions/upload-artifact@v4
-        with:
-          name: OrcaSlicer_dep_ubuntu_2404_${{ env.date }}
-          path: ${{ github.workspace }}/deps/build/OrcaSlicer_dep_ubuntu_*.tar.gz
-
   build_orca:
     name: Build OrcaSlicer
     needs: [build_deps]
     if: ${{ !cancelled() && !inputs.build-deps-only && (inputs.valid-cache == true && needs.build_deps.result == 'skipped') || (inputs.valid-cache != true && success()) }}
diff --git a/.github/workflows/build_orca.yml b/.github/workflows/build_orca.yml
--- .github/workflows/build_orca.yml
+++ .github/workflows/build_orca.yml
@@ -220,27 +220,20 @@
           max_releases: 1
 
 # Ubuntu
       - name: Install dependencies
-        if: inputs.os == 'ubuntu-20.04'
+        if: inputs.os == 'ubuntu-20.04' || inputs.os == 'ubuntu-24.04'
+        env:
+          apt-cmd: ${{ (inputs.os == 'ubuntu-20.04' && 'apt-fast') || (inputs.os == 'ubuntu-24.04' && 'sudo apt-get') || '' }}
+          webkit-ver: ${{ (inputs.os == 'ubuntu-20.04' && '4.0') || (inputs.os == 'ubuntu-24.04' && '4.1') || '' }}
         run: |
-          apt-fast update
-          apt-fast install -y autoconf build-essential cmake curl eglexternalplatform-dev \
+          ${{ env.apt-cmd }} update
+          ${{ env.apt-cmd }} install -y autoconf build-essential cmake curl eglexternalplatform-dev \
           extra-cmake-modules file git libcairo2-dev libcurl4-openssl-dev libdbus-1-dev libglew-dev libglu1-mesa-dev \
           libglu1-mesa-dev libgstreamer1.0-dev libgstreamerd-3-dev libgstreamer-plugins-base1.0-dev libgstreamer-plugins-good1.0-dev \
           libgtk-3-dev libgtk-3-dev libmspack-dev libosmesa6-dev libsecret-1-dev libsoup2.4-dev libssl-dev libudev-dev libwayland-dev \
-          libwebkit2gtk-4.0-dev libxkbcommon-dev locales locales-all m4 pkgconf sudo wayland-protocols wget
+          libwebkit2gtk-${{ env.webkit-ver }}-dev libxkbcommon-dev locales locales-all m4 pkgconf sudo wayland-protocols wget
 
-      - name: Install dependencies
-        if: inputs.os == 'ubuntu-24.04'
-        run: |
-          sudo apt-get update
-          sudo apt-get install -y autoconf build-essential cmake curl eglexternalplatform-dev \
-          extra-cmake-modules file git libcairo2-dev libcurl4-openssl-dev libdbus-1-dev libglew-dev libglu1-mesa-dev \
-          libglu1-mesa-dev libgstreamer1.0-dev libgstreamerd-3-dev libgstreamer-plugins-base1.0-dev libgstreamer-plugins-good1.0-dev \
-          libgtk-3-dev libgtk-3-dev libmspack-dev libosmesa6-dev libsecret-1-dev libsoup2.4-dev libssl-dev libudev-dev libwayland-dev \
-          libwebkit2gtk-4.1-dev libxkbcommon-dev locales locales-all m4 pkgconf sudo wayland-protocols wget libfuse2t64
-
       - name: Install dependencies from BuildLinux.sh
         if: inputs.os == 'ubuntu-20.04' || inputs.os == 'ubuntu-24.04'
         shell: bash
         run: sudo ./BuildLinux.sh -ur
@@ -267,40 +260,28 @@
           cd ${{ github.workspace }}/resources/profiles
           zip -r orca_custom_preset_tests.zip user/
 
       - name: Upload artifacts Ubuntu
-        if: ${{ ! env.ACT && inputs.os == 'ubuntu-20.04' }}
+        if: ${{ ! env.ACT && inputs.os == 'ubuntu-20.04' || inputs.os == 'ubuntu-24.04' }}
+        env:
+          ubuntu-ver: ${{ (inputs.os == 'ubuntu-20.04' && '2004') || (inputs.os == 'ubuntu-24.04' && '2404') || '' }}
         uses: actions/upload-artifact@v4
         with:
-          name: OrcaSlicer_Linux_${{ env.ver }}
+          name: OrcaSlicer_Linux_ubuntu_${{ env.ubuntu-ver }}_${{ env.ver }}
           path: './build/OrcaSlicer_Linux_${{ env.ver }}.AppImage'
-      - name: Upload artifacts Ubuntu
-        if: ${{ ! env.ACT && inputs.os == 'ubuntu-24.04' }}
-        uses: actions/upload-artifact@v4
-        with:
-          name: OrcaSlicer_Linux_ubuntu2404_${{ env.ver }}
-          path: './build/OrcaSlicer_Linux_${{ env.ver }}.AppImage'
 
       - name: Deploy Ubuntu release
-        if: ${{ ! env.ACT && github.ref == 'refs/heads/main' && inputs.os == 'ubuntu-20.04' }}
+        if: ${{ ! env.ACT && github.ref == 'refs/heads/main' && inputs.os == 'ubuntu-20.04' || inputs.os == 'ubuntu-24.04' }}
+        env:
+          ubuntu-ver-str: ${{ (inputs.os == 'ubuntu-24.04' && '_Ubuntu2404') || '' }}
         uses: WebFreak001/[email protected]
         with:
           upload_url: https://uploads.github.com/repos/SoftFever/OrcaSlicer/releases/137995723/assets{?name,label}
           release_id: 137995723
-          asset_path: ./build/OrcaSlicer_Linux_${{ env.ver }}.AppImage
+          asset_path: ./build/OrcaSlicer_Linux${{ env.ubuntu-ver-str }}_${{ env.ver }}.AppImage
           asset_name: OrcaSlicer_Linux_${{ env.ver }}.AppImage
           asset_content_type: application/octet-stream
           max_releases: 1 # optional, if there are more releases than this matching the asset_name, the oldest ones are going to be deleted
-      - name: Deploy Ubuntu release
-        if: ${{ ! env.ACT && github.ref == 'refs/heads/main' && inputs.os == 'ubuntu-24.04' }}
-        uses: WebFreak001/[email protected]
-        with:
-          upload_url: https://uploads.github.com/repos/SoftFever/OrcaSlicer/releases/137995723/assets{?name,label}
-          release_id: 137995723
-          asset_path: ./build/OrcaSlicer_Linux_${{ env.ver }}.AppImage
-          asset_name: OrcaSlicer_Linux_ubuntu2404_${{ env.ver }}.AppImage
-          asset_content_type: application/octet-stream
-          max_releases: 1 # optional, if there are more releases than this matching the asset_name, the oldest ones are going to be deleted
 
       - name: Deploy orca_custom_preset_tests
         if: ${{ ! env.ACT && github.ref == 'refs/heads/main' && inputs.os == 'ubuntu-20.04' }}
         uses: WebFreak001/[email protected]

Here are some suggestions that reduce the amount of repetitive code in the action files

@ElectricalBoy
Copy link
Contributor Author

Here are some suggestions that reduce the amount of repetitive code in the action files

Applied as 1e774b6. Thanks!

@ElectricalBoy
Copy link
Contributor Author

 -:  --------- >  1:  ee67b8d76 more missing includes (#5930)
 -:  --------- >  2:  1aa581762 update Ukrianian translations ( 2.1.1 )  (#5956)
 -:  --------- >  3:  770768506 Disable small area flow compensation on first layer (Fixes #4617) (#5838)
 1:  a682e6b86 =  4:  9450d8c00 CI: Build with ubuntu-24.04
 2:  836739d93 =  5:  34b7a49e9 Use apt-get in ubuntu-24.04
 3:  6f69afd72 =  6:  60327a6dd Fix step name
 4:  e4f57e6c6 =  7:  dc1c15cd8 Use alternate approach for detecting libwebkit2gtk version
 5:  1dd57a0ec =  8:  7b099205b CI: Install libfuse2 in ubuntu-24.04
 6:  1e774b6fb =  9:  530c17247 Reduce repetitive code in workflows
 7:  ad90f80d9 = 10:  a99bca2b6 CI: Install libfuse2
 8:  db5b54e23 = 11:  1768d4a51 Fix filenames for nightly deployment
 9:  013f0ad7e = 12:  5b2609ac5 Properly nest conditional

@SoftFever
Copy link
Owner

Looks great!
Thank you @ElectricalBoy and @Ocraftyone 🥇

@SoftFever SoftFever merged commit 0b6fc3d into SoftFever:main Jul 1, 2024
15 checks passed
@ElectricalBoy ElectricalBoy deleted the ubuntu-22.04-build branch July 1, 2024 22:28
@eblanshey
Copy link

eblanshey commented Jul 3, 2024

@SoftFever do we need a new release to have the 24.04 AppImage build available? Or can the new AppImage be built for the existing release?

@ElectricalBoy
Copy link
Contributor Author

ElectricalBoy commented Jul 3, 2024

@SoftFever do we need a new release to have the 24.04 AppImage build available? Or can the new AppImage be built for the existing release?

You can download the 24.04 AppImage from the latest GHA run for the time being: https://github.com/SoftFever/OrcaSlicer/actions/runs/9762226827/artifacts/1660012415

Edit: now the build is available in Nightly too

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
5 participants