Fix #95541: Portable build for Linux (AppImage)

[shoogle]

This adds a Makefile target called "portable" which packs MuseScore into a directory "MuseScore.AppDir". This directory can then be turned into a portable AppImage using the AppImageKit.

[lasconic]

Tests failed, probably because of your changes with INSTPREFIX and globalshare. Somehow MuseScore cannot find the chords xml files https://github.com/musescore/MuseScore/blob/master/libmscore/chordlist.cpp#L1702 I'm not sure if this INSTPREFIX changes is really necessary, but it will probably break other things.

[shoogle]

Hmm, globalshare needs to be relative for portable builds, and I removed INSTPREFIX because I couldn't find any references to it in the code other than by globalshare. I can probably solve this by either:

• Trying a relative globalshare and defaulting back to INSTPREFIX if it's not found.
• Or, only making globalshare relative for portable builds.

I'll have a look and see what works best.

[lasconic]

What about keeping INSTPREFIX and in the makefile use a relative CMAKE_INSTALL_PREFIX ?

[shoogle]

Unfortunately, if CMake is given a relative path it appears to resolve it to a full path immediately, before passing it on via the config file.

[lasconic]

that's a strong assumption no?

[shoogle]

I tested on Ubuntu, Fedora, openSUSE and CentOS and these were all available. I think distros are pretty good about including codecs (except non-free ones like MP3) so that media will "just work" out of the box.

I included a new command line options "check-depends" which will scan the system for included libraries. If someone has a problem with the AppImage we can ask them to do:
MuseScore*.AppImage check-depends > dependencies.txt
Then can upload the "dependencies.txt" file to the forum and we will know what they are missing.

[lasconic]

Ok. Except the assumptions on libsdnfile, ogg etc... I don't have other remarks.
So what else do we need in order to run make package on travis and have a distributable file?

[shoogle]

So one thing I should mention is that portaudio is the only audio backend included in the package. Pulse, ALSA and JACK are servers which need to be running in the background before MuseScore starts and so cannot (to the best of my rather limited knowledge in the area) be included in the package. However, at least one of those will be available in the default repository of pretty much every distro, and ALSA seems to be included by default on Ubuntu and openSUSE.

[lasconic]

I will check the audio libs. Jack, alsa, and pulse are indeed servers, but MuseScore does need the client libs to connect to these. If they are not in the package, it could be that the support for these backends will just be removed.

[shoogle]

Yes, the client libraries are in the package. We can decide how to handle the audio later, perhaps by detecting which server is present on the system automatically, but I think it would be good to get the AppImages out there now so that people can start testing them on a range of systems.

Regarding the build, it has to be done on CentOS 6.7 to guarantee widest compatibility, but it can be done inside a CentOS chroot on any other Linux (e.g. Ubuntu 12.04 if that is what Travis uses).

I think there are three ways it could be done:

1. Change "make revision" to "make portable" inside '.travis.yml' so that builds happen on each commit.
2. Use a custom build server like the Windows and Linux nightlies.
3. Use a separate Travis server like the examples at https://github.com/probonopd/AppImages

probonopd's examples use Docker to tell Travis to use CentOS instead of Ubuntu. The build is triggered by running this command on a local machine:

# For example, to build arduino:
bash <(curl -s https://raw.githubusercontent.com/probonopd/AppImages/master/trigger.sh) arduino

This then triggers the build to happen on the remote Travis machine, and Travis sends the resulting binary to a repository on Bintray. I could look into creating equivalent scripts for MuseScore and setting up a Bintray repository, unless you would prefer to keep it "in-house"? The trigger command could be run by the server doing the Mac or Windows nightlies.

Once inside CentOS (chroot or otherwise) the build process is the same:

branch=master # or "2.0.3" etc.
date="$(date +%Y-%m-%d-%H%M)"
# Get newer version of git than available by default
wget https://people.centos.org/tru/devtools-2/devtools-2.repo -O /etc/yum.repos.d/devtools-2.repo
yum install -y devtoolset-2-git
alias git=/opt/rh/devtoolset-2/root/usr/bin/git
# Build AppImageKit now to avoid conflicts with MuseScore's dependencies:
git clone -b master --single-branch https://github.com/probonopd/AppImageKit.git
cd AppImageKit
./build.sh
cd ..
# install MuseScore dependencies
yum install -y ... # I have a list of dependencies for CentOS 6.7
git clone -b $branch --single-branch https://github.com/musescore/MuseScore.git
cd MuseScore
make revision && revision="$(cat mscore/revision.h)"
app="MuseScoreNightly-$date-$branch-$revision" # "-$arch" is appended by 'CMakeLists.txt'.
make \
    PREFIX="$app" \
    SUFFIX="-portable-nightly" \
    LABEL="MuseScore Portable Nightly Build" \
    portable
appdir="$(cat build.release/PREFIX.txt)"
appimage="$(echo "$appdir" | sed 's/AppDir$/AppImage/')"
cd ../AppImageKit/AppImageAssistant.AppDir
./package  "$appdir" "$appimage"
# upload $appimage to Bintray, musescore.org or wherever.

[shoogle]

So running that code on a 64 bit machine would build a file called:

MuseScoreNightly-2016-01-29-1729-master-50d6898-x86_64.AppImage


Just running "make portable" (without overriding PREFIX, SUFFIX and LABEL) would result in:

MuseScore-x86_64.AppImage


Or, if $UNSTABLE is FALSE inside CMakeLists.txt then the version is added automatically, like this:

MuseScore-2.0.3-x86_64.AppImage


[lasconic]

Ok. That's a lot so let me go point by point.

Travis is the way to go. I would propose to use the MuseScore repository. Currently travis builds on Ubuntu MuseScore and the tests, run the tests, and upload images on amazon S3. We should indeed use Docker to build it on CentOS. However we don't need to do all the things from the script you wrote. Source code is already on Travis for example. Also, we don't want the build on CentOS to come after the build on Ubuntu, it would be great to have them run in parallel. We could take advantage of this https://docs.travis-ci.com/user/speeding-up-the-build/. A script would check for an environment variable and run the right build (ubuntu for the tests, or centos for the portable app continuous delivery)

In order to put this in place, it's probably better to activate Travis on your MuseScore fork, create a branch and modify the travis configuration file for there. I'm on IRC to help if needed, I have some experience with docker and travis.

For the last step, we have a sftp access to the OSUOSL share where we store the releases and the nightlies for mac and windows. I guess it would make sense to push the portable app there as well. Once we have the whole setup running I will ask OSUOSL for the best way to do this. If it doesn't work, sure bintray is possible (I just created an account, just in case... https://bintray.com/musescore/)

[probonopd]

@shoogle thanks for your great work! Can you upload your AppImage somewhere for testing? I'd like to run it though a couple of different distributions to check whether all required libraries are bundled in a way that maximizes compatibility with different host systems. Thanks.

[shoogle]

@probonopd no problem! Here's a link. If you're running it on different distros perhaps you could also run MuseScore*.AppImage check-depends and let me know if any libraries are missing on the distro? I've already checked Fedora 21 & 22, Ubuntu 12.04 & 15.10, openSUSE 13.1 and Centos 6.7. Is there a way to test it on Arch Linux?

[ericfont]

I just tried on my Arch Linux x86-64, and works (but haven't done any testing other than open one of my scores and play audio). Amazing. Now of course I already have musescore official arch packag installed, so I would already have all the libaries. Did you want a test on a clean install that wouldn't already have libraries?

[ericfont]

attached is output:

./MuseScoreDevelopment-80860da-x86_64.AppImage check-depends
AppImage contains 561 executables and 80 libraries.
Checking dependencies for executables and libraries...
Processing results.

app-archx86-64.txt

[probonopd]

Fedora 23 Fedora-Live-Workstation-x86_64-23-10.iso (out-of-the-box; MuseScore never installed previously)

Runs, but prints on the console

QObject::connect: Cannot connect (null)::stateChanged(QNetworkSession::State) to QNetworkReply
HttpImpl::_q_networkSessionStateChanged(QNetworkSession::State)
(repeated a couple of times)

QNetworkReplyImplPrivate::error: Internal problem, this method must only be called once.
QObject::connect: Cannot connect (null)::stateChanged(QNetworkSession::State) to QNetworkReplyHttpImpl::_q_networkSessionStateChanged(QNetworkSession::State)

check-depends says AppImage contains 561 executables and 80 libraries. 561 executables, I think the permissions are wrong inside the AppImage.

fedora23.txt

You could use a higher-resolution icon to make it less pixelated in the Gnome 3 Files icon view.

[probonopd]

CentOS 7 CentOS-7.0-1406-x86_64-GnomeLive.iso (out-of-the-box; MuseScore never installed previously)

Runs but in addition to the output above says

initScoreFonts 0x315c650
init Help from: </tmp/.mount_3Sy2jH/share/mscore-portable-dev-2.1/manual/doc_de_DE.qhc>
cannot setup data for help engine: Cannot open collection file: /tmp/.mount_3Sy2jH/share/mscore-portable-dev-2.1/manual/doc_de_DE.qhc

centos7.txt

[probonopd]

Antergos antergos-2014.08.07-x86_64.iso (out-of-the-box; MuseScore never installed previously)

Runs with the usual messages

antergos.txt

[shoogle]

Many thanks for these.

@ericfont I have an Arch ISO and a spare partition all ready to go so I'll give it a try myself (though I did wonder if @probonopd knew a way of testing Arch without installing the ISO since that is possible for most of the other distros).

@probonopd the large number of executables is indeed due to incorrect permissions, but I think this is actually caused by a bug in AppImageKit. If I run the test on the AppDir (before conversion into an AppImage) then only 5 executables are found. It appears that AppImageKit sets execute permission to everything in the "lib" folder upon conversion to an AppImage, even though libraries on Linux don't usually need execute permission.

The dependency output from CentOS 7 (centos7.txt) is very strange. Ironically, it appears that the ldd-recursive script I use with check-depends is missing one of its own dependencies on CentOS 7.

Regarding the Qt error messages at runtime, I don't really know much about those but perhaps @lasconic could shed some light on the matter? I sometimes see similar messages on Ubuntu using the official packages, but everything appears to work fine regardless. When checking on the different distros all I tested was:

• Does MuseScore itself run?
• Do the plugins appear in the Plugins menu after loading from the Plugin Manager?
• Are the templates found if you go to File -> New -> Next?
• Are the web components in the Start Centre visible?
• Is there sound when using PortAudio? (Or with Pulse, ALSA or JACK if installed?)
• Are all dependencies listed in MuseScore*.AppImage check-depends available?

As far as I am aware, if those things are all true then all is good?

[ericfont]

Antergos is an Arch Linux distro that you can run live without installing the ISO. There are others. Though none are considered offical by Arch officials.

[shoogle]

@ericfont I see, thanks.

@probonopd Regarding the icon (.DirIcon), we have SVGs so I can easily generate a high resolution PNG if required, but is it possible to use the SVG itself as the icon?

[probonopd]

@shoogle Yes, it is possible to use an SVG as an icon.

[shoogle]

@probonopd I know it is possible when installing the conventional way, but how can I tell AppImageKit to use it instead of a PNG? When I tried before it couldn't find it.

[probonopd]

@shoogle e.g., put Icon=musescore into musescore.desktopand then put musescore.svg into the root of the AppDir.

[shoogle]

@probonopd it still seems to be finding the PNGs first. Do I have to delete them?

@lasconic I've got something working here: https://github.com/shoogle/MuseScore/tree/portable-linux-build-on-travis. Have a look at my .travis.yml and the new file build/Linux+BSD/portable/Recipe.

• The Ubuntu debug build and the AppImage build take place in parallel on separate Travis VMs.
• The Ubuntu dependencies are only loaded for the Ubuntu build.
• The AppImage build simply mounts the MuseScore directory inside a Docker image and then runs the recipe.
• The AppImage is created at /MuseScore/build.release/MuseScore*.AppImage.

I'll need your help with the upload to S3.

Not essential right now, but at some point I'll need to know what you want to happen regarding:

• Build failures (I set it so the AppImage build is allowed to fail but the Ubuntu build is not)
• Notifications (what shall we do if the AppImage build fails?)
• Nightly splash image (include it in the source code or "wget it" from somewhere else?)
• Procedure for official releases (presumably you would like to push a commit with UNSTABLE=FALSE inside CMakeLists.txt and then everything else would be automatic? i.e. all references to "Nightly Build" would be ignored, etc.)

[shoogle]

Re. official releases, other points to consider will be:

• Builds for different architectures (32bit & ARM).
  • What to do if we need to release a patched AppImage in between MuseScore releases email (e.g. MuseScore-2.0.2-3-x86-64.AppImage)

[ericfont]

How long did it take you to setup this appimage? I could try building an ARM myself now to test out.

[shoogle]

It appears there isn't an ARM version of CentOS 6, but I'll try to generalise my scripts so that they should run on CentOS 7 and outside of Docker. Ideally we would build on the oldest version of Linux available to maximise compatibility, but it's not worth compiling all the dependencies ourselves.

[probonopd]

@shoogle Yes, currently png take precedence over svg iirc but if you place an svg (and no png) with the correct name in the top-level AppDir then it should be found and used. Let me know if that is not the case.

[shoogle]

@lasconic Done! I just tested it on my own branch too and confirmed that the AppImage uploaded to Bintray correctly.

[shoogle]

Added the nightly splash image as promised. Bintray upload confirmed successful via my own branch.

[lasconic]

Ok ! Let's merge it !

[lasconic]

Ok it's building. In the meantime, a couple of things to improve for the next iteration.

1. https://bintray.com/musescore/nightlies-linux. We should probably create other packages here for 32 bit, arm etc... and not use the existing package. The existing package MuseScoreNightly-portable-linux-final-tests probably needs a better name and mention x86_64. This way we can take advantage of the "latest" badge of bintray.
2. If it's smooth with the nightlies, we will use AppImage for the release too. It would probably make sense to activate the checkForUpdate() function for these packages then.

[ericfont]

I'm reading about bintray and skimming its manaul now, but there I still don't understand: Does bintray itself compile the binaries from your github, or does the compilation happen elsewhere and is then uploaded to bintray? Is it able to compile and publish a nightly binary every time you commit? And I would presume the build infrastructure is x86 only. So my next question is would it be for building an arm portable is it better to setup an x86->arm cross compiler script which to build inside whatever online x86 virtual machine is available, or would it be better to build on an arm machine and then upload?

(As it is, I'm able to build mscore portable armv6 on my Raspberry running Jessie, but need to finalise the copy-libs scripts. I am having some issues building portable on Raspbian Wheezy, due to unavilablity of qt5 binaries, meaning I have to use my own compiled qt5 libraries...)

[lasconic]

Bintray is just a bin tray. So it's glorified storage. Compilation currently happens on Travis. Travis is pinged for every new commits made on github and it compiles MuseScore and runs the tests. It does it for your Pull Requests too btw. That's how we know if the tests passes or not.

I want to use the same indrastructure for ARM or for 32 bit, we will indeed need to cross compile. Travis currently run Ubuntu 12.04 64 bits. Trusty 14.04 64 bits is in beta.

[ericfont]

ok, thanks for prompt reply. I will look into using cross compiler...probably this gcc 4.9.3 here https://github.com/raspberrypi/tools/ for building a portable using the x86-64 travis ubuntu 12.04. Note, this might require pre-compiled binary arm libraries which the build script would simply download (either I could provide my self-compiled qt5.5.1, or maybe just extract from Raspbian Jessie), so copy-libs will simply copy. (trying to cross-compile qt-libraries for every build would be far too time consuming.)

[lasconic]

We already do this for x86_64 since 12.04 doesn't have up to date Qt. However, we currently use the latest Qt 5.4 for MacOSX, Windows and Linux Tests (see https://github.com/musescore/MuseScore/blob/master/build/travis/job1_Tests/environment.sh#L20)

Apparently for the AppImage @shoogle is using the one available on centOS https://github.com/musescore/MuseScore/blob/master/build/Linux%2BBSD/portable/Recipe#L79 maybe we should consider using the same version we use for tests, mac and windows instead.

[shoogle]

@lasconic Regarding your comment on things to improve for next time:

1. One step ahead of you here ;) New packages are automatically created by "bintray.sh" and named "MuseScoreNightly-<branch>". My test branch was called "portable-linux-final-tests", hence where that package came from (which I will delete shortly). The nightlies are already in the correct packages for their branches. The package contains subfolders for each architecture, so the latest version badge should work even if 64 bit and 32 bit are in the same package. I put a dummy 32 bit package in the test package so you can see how it works. Unfortunately it doesn't appear to work quite as expected if there is a latest version on one architecture but not another. It works fine once the latest version is available for both architectures, but perhaps you are right and we should name package instead.
   .
   Edit: I've now deleted the test package, but I discovered a way to link directly to the latest version of the list of files - Simply add "#files" directly after "_latestVersion" in the link (i.e. don't add "/view#files" as I was trying to do and it wasn't working)
2. That's a good idea. It might even be possible to get the packages to update themselves automatically using the zsync files (after asking the user if they would like to do so).

[shoogle]

Regarding using the same Qt as for the tests (i.e. 5.4), this is certainly worth keeping an eye on for the future, but it may not be as simple as it sounds at present because:

• Qt 5.5 fixes various bugs on Linux (especially on ARM)
• The CentOS Qt seems to be better optimised (it's slightly smaller than the generic Qt and has fewer dependencies.)
• There isn't a generic Qt for ARM, only distribution packages.

@ericfont To see what my code does, including the Bintray upload, the best thing to do is skim read the .travis.yml and this build.log. Ignore the build labelled "JOB=Tests" (that's the normal Travis build which runs tests on Ubuntu) and instead click on the "JOB=AppImage" build so see it's log (skim read!!!). Also read in more detail the files referenced in the "JOB=AppImage" section of .travis.yml (i.e. build.sh, and files it references: bintray.sh). Any questions, please post elsewhere (Github or MuseScore forum) and then post a link here for us to follow.

[shoogle]

PR for 32 bit AppImages: #2398

[lasconic]

On Windows and on MacOSX, we sign the install packages and the binaries. Is there anything similar that could/should be done on Linux?

[shoogle]

It's not as essential as on Windows and Mac where I believe it helps avoid issues with overly zealous antivirus software, but it would be nice. There's not a centralised system for all Linux, but @probonopd seems to have some kind of PGP signing going on with these .asc files. It looks like this is a feature of Bintray rather than something done manually. Another option would be simply to publish MD5 checksums somewhere on musescore.org.

However, the vast majority of users won't bother verifying the signatures, so we won't see any real benefit until the infrastructure for automatic updates is in place.

[lasconic]

We also publish sums for windows/mac packages, so we will do that. Not sure the linux kernel is up to verifying signature of binaries.

[probonopd]

AppImageUpdate will check for signatures.

[ericfont]

All I see on that AppImageUpdate readme.md is: "Of course, I want to check with GPG signatures where the update is coming from"...I haven't looked deeper, but my question is: Is there anything that needs to be added to musescore's code to setup this signature checking on update? What about for the initial execution of AppImage? Although, I suppose the github -> travis -> bintray -> download is a secure ssl chain...if someone is downloading from some random http site then I guess that is their risk.

[shoogle]

As far as the initial run is concerned, the user must do the test themself. We can't program a signature check into the AppImage itself because then a malicious AppImage could just print "Test passed!" without actually checking anything, not to mention the difficulty in knowing what the signature will be before the AppImage is created. I suppose we could include signature checking on update, if updating was done within MuseScore. So it depends on whether we want to create our own update mechanism or just have MuseScore check for updates and then rely on something else for actually doing the update (i.e. AppImageupdate).

I should also mention that I added some new command line options specifically for the AppImage. One of them is install, which copies icon and mime information to the host system for desktop integration:

MuseScore*.AppImage  install

We'd need a way to update the installed files if they have changed in the newer version of the AppImage, and we'd need a way to delete them if the AppImage itself is deleted. At present the user must run:

MuseScore*.AppImage  remove

These are all questions that @probonopd has been considering for some time with the AppImage format. My personal take is that a centralized AppImageManager for installing, updating and removing AppImages would be great, but they should still be fully usable on their own (hence the command line options to install and remove).

[lasconic]

It's probably not the right place to discuss this, so can you mention a url where @probonopd has been considering this?
My take would be more MacOSX like... the AppImageManager would be transparent to the user but when a AppImage is installed (added on the device, or maybe run) the manager should detect it and run a specific install option for example and verify signatures. In the same way, when the .AppImage is deleted (or even sent to the trash), the manager should detect it and run the "remove" option to delete file assocation etc... If the manager is another package manager... it makes AppImage less relevant, not more...

[ericfont]

(I'm sorry for sidetracking...I brought it up because was just reading about ransomware hack of transmission bittorrent installer.)

Regarding the initial run, maybe the official musescore nightly linux download page could include instructions for manual verification, and maybe a warning about to not trust http or random url downloads.

Regarding checking signature on update: I think MuseScore AppImage should check for updates and if user accepts, then runs command for AppImageUpdate including signature verification.

[ericfont]

@shoogle, I'm getting a crash when try to run any plugin from latest bintray AppImage on my x86-64 Parabola (Arch) Linux machine: https://musescore.org/en/node/101101#comment-450641

On a related subject, since the portable app image strips the debug symbols, that means it is now very difficult to investigate crashes because there is no stack trace, and gdb isn't very useful without debug symbols. I suppose I could make appimage to investigtate. or is there something easier I should try first?

It would be nice if we have travis automatically generate AppImage with debug symbols included... Storage space on bintray is free...

EDIT: I build the AppImage as DEBUG and reproduced the crash. I made new issue, including backtrace, here: https://musescore.org/en/node/101171

[probonopd]

@lasconic one core idea of AppImage is that it does not need any "runtime" to be installed on the base system, e.g., no "AppImageManager" is mandatory. AppImageUpdate is a purely optional component. It grabs the URL that is embedded inside the AppImage, looks for the latest version there, and downloads the binary delta. It is still relatively new, so if you have ideas how to make it better, feel free to open issues and/or send PRs to AppImageKit.

Of course, an app that is packaged inside an AppImage could run some checks; but as pointed out before a malicious AppImage could just pretend the checks have run successfully. Hence, whatever checks the signature of the AppImage needs to be a "trusted" application, whatever that means for the respective user.