Update SABnzbd to 4.3.2 #6107

Safihre · 2024-05-29T20:03:18Z

Description

Update SABnzbd to 4.3.2

Checklist

Build rule all-supported completed successfully
New installation of package completed successfully
Package upgrade completed successfully (Manually install the package again)
Package functionality was tested
Any needed documentation is updated/created

Type of change

Safihre · 2024-05-29T20:05:31Z

@th0ma7 I was a bit surprised it tries to build Python for 88f6281 (which fails), even though SABnzbd specifies UNSUPPORTED_ARCHS = $(ARMv5_ARCHS) $(OLD_PPC_ARCHS).

th0ma7 · 2024-05-29T21:15:09Z

I was a bit surprised it tries to build Python for 88f6281 (which fails), even though SABnzbd specifies UNSUPPORTED_ARCHS = $(ARMv5_ARCHS) $(OLD_PPC_ARCHS).

Actually there's two totally different problems:

I thought @hgy59 had solve buildig py311 for armv5, I've re-started the github build as it may just be a "one of"
in the .github/action/* directory there's a use-case missing where if there isn't any package to be built for this arch, then don't even bother to build the perequisite package that where included in the build list (such as python or ffmpeg)... Unless it has to be managed at the matrix step from the .github/workflows/build.yml. I believe it's @publicarray who wrote that part of the code, he may be able to provide hints on that?

hgy59 · 2024-05-29T21:36:35Z

@th0ma7 in #6094 I found that ARMv5 fails to build some rust binaries. This is caused by rust >= 1.78 that has a hard dependency on libatomic.
It fails to build meson too, required by python (crossenv).

What shall we do? Should we nail rust to the latest 1.77.* version for all ARMv5 builds?

th0ma7 · 2024-05-29T22:25:18Z

in #6094 I found that ARMv5 fails to build some rust binaries. This is caused by rust >= 1.78 that has a hard dependency on libatomic. It fails to build meson too, required by python (crossenv).

What shall we do? Should we nail rust to the latest 1.77.* version for all ARMv5 builds?

Nice catch! Indeed in spksrc.cross-rust-env.mk exists RUSTUP_DEFAULT_TOOLCHAIN = stable which could be set to a version specific under the ARMv5 definition.

Further looking at the code the spksrc.tc-rust.mk needs to make usage of that variable (current hardcoded as rustup override set stable).

That should be relatively easy to do. I can try to create a quick PR for testing.

EDIT: Testing a potential begining of a fix in PR #6108

Safihre · 2024-05-30T11:27:08Z

Well, ARM v5 won't work for Sab anyway because unrar doesn't support it anymore. It needs C++11 minimum.

th0ma7 · 2024-05-30T11:31:11Z

Indeed but ideally we can still resolve the py311 build issue for it so checks are all green. My new pr should solve that hopefully. Le jeu. 30 mai 2024, 07 h 27, Safihre ***@***.***> a écrit :

…

Well, ARM v5 won't work for Sab anyway because unrar doesn't support it anymore. It needs C++11 minimum. — Reply to this email directly, view it on GitHub <#6107 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AECZA2SKXAX6CNT5RWKVZCLZE4EKFAVCNFSM6AAAAABIPTMCCWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMZZGM2DQOBXG4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

th0ma7 · 2024-05-30T21:40:54Z

@Safihre would you mind rebasing against master now that my PR python311 fix is merged for armv5 no-atomic? much thnx in advance.

Safihre · 2024-05-31T08:24:19Z

@th0ma7 worked!

Safihre · 2024-05-31T20:22:24Z

Strange, when I rand the build to publish it suddenly failed for qoriq even though here it succeeded?!
https://github.com/Safihre/spksrc/actions/runs/9320470672

th0ma7 · 2024-05-31T21:41:08Z

Sometimes it is just because it completed at the same time as another one and parallel uploading fails on our package server. Just re-run failed ones.

mreid-tt · 2024-05-31T22:28:25Z

@th0ma7 I don't think the issue is the same as a failed upload. The build fails to complete in the first place:

2024.05.31 15:40:17 - python311: (qoriq-6.2.4) FAILED
2024.05.31 15:44:29 - sabnzbd: (qoriq-6.2.4) FAILED

Based on the logs I see this:

2024-05-31T15:41:22.3389732Z       running build_rust
2024-05-31T15:41:22.3390676Z       error: override toolchain 'stage2-powerpc-unknown-linux-gnuspe' is not installed: the RUSTUP_TOOLCHAIN environment variable specifies an uninstalled toolchain
2024-05-31T15:41:22.3391539Z       
2024-05-31T15:41:22.3391852Z           =============================DEBUG ASSISTANCE=============================
2024-05-31T15:41:22.3392434Z           If you are seeing a compilation error please try the following steps to
2024-05-31T15:41:22.3392982Z           successfully install cryptography:
2024-05-31T15:41:22.3393600Z           1) Upgrade to the latest pip and try again. This will fix errors for most
2024-05-31T15:41:22.3394335Z              users. See: https://pip.pypa.io/en/stable/installing/#upgrading-pip
2024-05-31T15:41:22.3394984Z           2) Read https://cryptography.io/en/latest/installation/ for specific
2024-05-31T15:41:22.3395515Z              instructions for your platform.
2024-05-31T15:41:22.3396001Z           3) Check our frequently asked questions for more information:
2024-05-31T15:41:22.3396512Z              https://cryptography.io/en/latest/faq/
2024-05-31T15:41:22.3396987Z           4) Ensure you have a recent Rust toolchain installed:
2024-05-31T15:41:22.3397505Z              https://cryptography.io/en/latest/installation/#rust
2024-05-31T15:41:22.3397921Z       
2024-05-31T15:41:22.3398160Z           Python: 3.11.5
2024-05-31T15:41:22.3398514Z           platform: Linux-powerpc-with
2024-05-31T15:41:22.3398865Z           pip: n/a
2024-05-31T15:41:22.3399140Z           setuptools: 70.0.0
2024-05-31T15:41:22.3399687Z           setuptools_rust: 1.9.0
2024-05-31T15:41:22.3400021Z           rustc: n/a
2024-05-31T15:41:22.3400369Z           =============================DEBUG ASSISTANCE=============================
2024-05-31T15:41:22.3400772Z       
2024-05-31T15:41:22.3401068Z       error: can't find Rust compiler
2024-05-31T15:41:22.3401501Z       
2024-05-31T15:41:22.3402482Z       If you are using an outdated pip version, it is possible a prebuilt wheel is available for this package but pip is not able to install from it. Installing from the wheel would avoid the need for a Rust compiler.
2024-05-31T15:41:22.3403598Z       
2024-05-31T15:41:22.3403840Z       To update pip, run:
2024-05-31T15:41:22.3404132Z       
2024-05-31T15:41:22.3404415Z           pip install --upgrade pip
2024-05-31T15:41:22.3404738Z       
2024-05-31T15:41:22.3405004Z       and then retry package installation.
2024-05-31T15:41:22.3405355Z       
2024-05-31T15:41:22.3406696Z       If you did intend to build this package from source, try installing a Rust compiler from your system package manager and ensure it is on the PATH during installation. Alternatively, rustup (available at https://rustup.rs) is the recommended way to download and update the Rust compiler toolchain.
2024-05-31T15:41:22.3408112Z       
2024-05-31T15:41:22.3408377Z       This package requires Rust >=1.56.0.
2024-05-31T15:41:22.3408742Z       [end of output]
2024-05-31T15:41:22.3409012Z   
2024-05-31T15:41:22.3409473Z   note: This error originates from a subprocess, and is likely not a problem with pip.
2024-05-31T15:41:22.3410087Z   ERROR: Failed building wheel for cryptography

I don't know if this is a setup issue but it would be useful @Safihre to re-run this failed job to see if a second run has the same issue.

th0ma7 · 2024-06-01T13:15:16Z

This is odd, further as it worked initially on two pr. @Safihre can you re-run failed build?

Safihre · 2024-06-01T13:43:55Z

Unfortunately, same result
https://github.com/Safihre/spksrc/actions/runs/9320470672/job/25684361784#step:7:20

th0ma7 · 2024-06-01T15:03:38Z

I'll try to have a look at it over the weekend.

th0ma7 · 2024-06-03T19:48:39Z

This is odd, looking at the github-action build logs, this should normally be the trigger for qoriq toolchain:

===>  Target powerpc-unknown-linux-gnuspe unavailable...

When this is seen normally shortly thereafter it should trigger the dependency processing for the toolchain such as:

===>  Processing dependencies of syno-qoriq
===>  Downloading files for rust-qoriq
===>    wget --secure-protocol=TLSv1_2 --timeout=30 --waitretry=0 --tries=5 -nv https://github.com/SynoCommunity/spksrc/releases/download/toolchains%2Frust%2F1.77.0-nightly/stage2_powerpc-unknown-linux-gnuspe.txz
2024-06-03 19:33:44 URL:https://objects.githubusercontent.com/github-production-release-asset-2e65be/2565137/e6d63626-804a-429a-a5a1-414e20a13e00?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20240603%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240603T193336Z&X-Amz-Expires=300&X-Amz-Signature=08a21a1b5afca29fd056be433666d6d970f2e710e144875e0788abbcb5071952&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=2565137&response-content-disposition=attachment%3B%20filename%3Dstage2_powerpc-unknown-linux-gnuspe.txz&response-content-type=application%2Foctet-stream [102497448/102497448] -> "stage2_powerpc-unknown-linux-gnuspe.txz.part" [1]
===>  Verifying files for rust-qoriq
===>    Checking sha1sum of file stage2_powerpc-unknown-linux-gnuspe.txz
===>    Checking sha256sum of file stage2_powerpc-unknown-linux-gnuspe.txz
===>    Checking md5sum of file stage2_powerpc-unknown-linux-gnuspe.txz
/home/spksrc/armv5-rust/spksrc/native/rust-qoriq/../../distrib/stage2_powerpc-unknown-linux-gnuspe.txz
===>  Processing dependencies of rust-qoriq
===>  Extracting for rust-qoriq
tar -xpf /home/spksrc/armv5-rust/spksrc/native/rust-qoriq/../../distrib/stage2_powerpc-unknown-linux-gnuspe.txz -C /home/spksrc/armv5-rust/spksrc/native/rust-qoriq/work-native/install//usr/local
===>  Patching for rust-qoriq
===>  Installing for rust-qoriq
find /home/spksrc/armv5-rust/spksrc/native/rust-qoriq/work-native/install//usr/local/ \! -type d -printf '%P\n' | sed 's?^target/??g' | sort > /home/spksrc/armv5-rust/spksrc/native/rust-qoriq/work-native/rust-qoriq.plist.tmp
export PATH="/home/spksrc/armv5-rust/spksrc/distrib/cargo/bin:${PATH}" ; \
export RUSTUP_HOME="/home/spksrc/armv5-rust/spksrc/distrib/rustup" ; \
which rustup ; \
rustup toolchain link stage2-powerpc-unknown-linux-gnuspe /home/spksrc/armv5-rust/spksrc/native/rust-qoriq/work-native/install/usr/local ; \
rustup show
/home/spksrc/armv5-rust/spksrc/distrib/cargo/bin/rustup
Default host: x86_64-unknown-linux-gnu
rustup home:  /home/spksrc/armv5-rust/spksrc/distrib/rustup

installed toolchains
--------------------

stable-x86_64-unknown-linux-gnu (default)
1.77.2-x86_64-unknown-linux-gnu
stage2-powerpc-unknown-linux-gnuspe

active toolchain
----------------

stable-x86_64-unknown-linux-gnu (default)
rustc 1.78.0 (9b00956e5 2024-04-29)

find /home/spksrc/armv5-rust/spksrc/native/rust-qoriq/work-native/install//usr/local// \! -type d -printf '%P\n' | sed 's?^target/??g' | sort | \
  diff /home/spksrc/armv5-rust/spksrc/native/rust-qoriq/work-native/rust-qoriq.plist.tmp -  | grep '>' | sed 's?> ??g' > /home/spksrc/armv5-rust/spksrc/native/rust-qoriq/work-native/rust-qoriq.plist

But that is entirely skipped somehow... I will have to figure out how to replicated this.

@hgy59 I'm running LXC/LXD on my end, are you runing docker and may be able to replicated this? which may be, again, related to the /spksrc or /worspace root directory? Easy way to tell is, from the master branch, just run make arch-qoriq-6.2.4 from for spk/python311 directory (or using -C argument)

EDIT: I may have found where the issue may be ... IF the native/rust-qoriq is being cached, it won't reprocess it again, therefore not re-installing the rust toolchain ... Fix could be:

do not cache native - not a good idea
move the symlinking out of native/rust-qoriq and down into toolchain/syno-qoriq-6.2.4/Makefile as POST_DEPEND_TARGET
enforce deletion of .rust-qoriq-install_done at the end of native/rust-qoriq
... any other idea?

hgy59 · 2024-06-03T22:26:48Z

EDIT: I may have found where the issue may be ... IF the native/rust-qoriq is being cached, it won't reprocess it again, therefore not re-installing the rust toolchain ... Fix could be:

related to #6054 ?

th0ma7 · 2024-06-03T23:39:26Z

@hgy59 yup, i believe this is the case...

th0ma7 · 2024-06-04T00:33:49Z

@Safihre potential fix now merged on master #6123

Can you update your branch and re-launch a build and see how things turns out for qoriq?

Safihre · 2024-06-04T20:59:23Z

Worked!
https://github.com/Safihre/spksrc/actions/runs/9373704014

Thanks a lot for your help every time! 👍👍

th0ma7 · 2024-06-04T21:19:41Z

Awesome! 🚀 Now let's hope it suffice for all github-action caching scenarios...

mreid-tt · 2024-06-04T23:17:16Z

@Safihre will you be publishing the qoriq package to the repo now that it's working?

Safihre · 2024-06-05T05:29:06Z

Oh thought it was an unsupported arch 😱

Safihre · 2024-06-05T08:16:13Z

Oh now that I trigger a build with publishing it failed again:
https://github.com/Safihre/spksrc/actions/runs/9141936527/job/25136968455
Cache? I can try to run it from master instead of the branch, since the code is in there.

mreid-tt · 2024-06-05T09:47:24Z

@Safihre, I've triggered a run on your behalf and it built and published successfully: https://github.com/mreid-tt/spksrc/actions/runs/9381360715/job/25830726706

th0ma7 · 2024-06-05T10:05:44Z

That's something to keep an eye on wetter it fixes things for good or not for qoriq building. Ping me if you notice this again.

Safihre · 2024-06-05T11:55:16Z

@Safihre, I've triggered a run on your behalf and it built and published successfully: https://github.com/mreid-tt/spksrc/actions/runs/9381360715/job/25830726706

Sounds like a cache problem. Thanks!

Safihre requested a review from th0ma7 May 29, 2024 20:03

Update SABnzbd to 4.3.2

9594c15

Safihre force-pushed the sabnzbd432 branch from 0ca0341 to 9594c15 Compare May 31, 2024 06:55

th0ma7 approved these changes May 31, 2024

View reviewed changes

Safihre merged commit 5eccb90 into SynoCommunity:master May 31, 2024
17 checks passed

th0ma7 mentioned this pull request Jun 4, 2024

rust-qoriq: Migrate from native to toolchain/syno-qoriq-6.2.4-rust #6123

Merged

10 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update SABnzbd to 4.3.2 #6107

Update SABnzbd to 4.3.2 #6107

Safihre commented May 29, 2024

Safihre commented May 29, 2024

th0ma7 commented May 29, 2024

hgy59 commented May 29, 2024

th0ma7 commented May 29, 2024 •

edited

Loading

Safihre commented May 30, 2024

th0ma7 commented May 30, 2024 via email

th0ma7 commented May 30, 2024

Safihre commented May 31, 2024

Safihre commented May 31, 2024

th0ma7 commented May 31, 2024

mreid-tt commented May 31, 2024 •

edited

Loading

th0ma7 commented Jun 1, 2024

Safihre commented Jun 1, 2024

th0ma7 commented Jun 1, 2024

th0ma7 commented Jun 3, 2024 •

edited

Loading

hgy59 commented Jun 3, 2024

th0ma7 commented Jun 3, 2024

th0ma7 commented Jun 4, 2024

Safihre commented Jun 4, 2024

th0ma7 commented Jun 4, 2024

mreid-tt commented Jun 4, 2024

Safihre commented Jun 5, 2024

Safihre commented Jun 5, 2024

mreid-tt commented Jun 5, 2024

th0ma7 commented Jun 5, 2024

Safihre commented Jun 5, 2024

Update SABnzbd to 4.3.2 #6107

Update SABnzbd to 4.3.2 #6107

Conversation

Safihre commented May 29, 2024

Description

Checklist

Type of change

Safihre commented May 29, 2024

th0ma7 commented May 29, 2024

hgy59 commented May 29, 2024

th0ma7 commented May 29, 2024 • edited Loading

Safihre commented May 30, 2024

th0ma7 commented May 30, 2024 via email

th0ma7 commented May 30, 2024

Safihre commented May 31, 2024

Safihre commented May 31, 2024

th0ma7 commented May 31, 2024

mreid-tt commented May 31, 2024 • edited Loading

th0ma7 commented Jun 1, 2024

Safihre commented Jun 1, 2024

th0ma7 commented Jun 1, 2024

th0ma7 commented Jun 3, 2024 • edited Loading

hgy59 commented Jun 3, 2024

th0ma7 commented Jun 3, 2024

th0ma7 commented Jun 4, 2024

Safihre commented Jun 4, 2024

th0ma7 commented Jun 4, 2024

mreid-tt commented Jun 4, 2024

Safihre commented Jun 5, 2024

Safihre commented Jun 5, 2024

mreid-tt commented Jun 5, 2024

th0ma7 commented Jun 5, 2024

Safihre commented Jun 5, 2024

th0ma7 commented May 29, 2024 •

edited

Loading

mreid-tt commented May 31, 2024 •

edited

Loading

th0ma7 commented Jun 3, 2024 •

edited

Loading