Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

QDirStat AppImage and Why I don't like It #168

Closed
shundhammer opened this issue Jun 21, 2021 · 13 comments
Closed

QDirStat AppImage and Why I don't like It #168

shundhammer opened this issue Jun 21, 2021 · 13 comments

Comments

@shundhammer
Copy link
Owner

shundhammer commented Jun 21, 2021

QDirStat AppImage

TL;DR

  • It's big and fat (113 MB packed, 586 MB unpacked)
  • It's a very outdated version: QDirStat 1.6.1 February 2020
  • It doesn't even tell you what version it contains
  • It still needs a fairly recent version of GLibc, so you can't run it on older (but still supported!) Linux distributions like Ubuntu 18.04 LTS
  • It's unclear if at least the libraries inside (e.g. the Qt libs) are up to date

Details

I recently came across this:

https://apprepo.de/appimage/qdirstat

QDirStat-AppImage-page

Okay... so somebody packaged QDirStat as an AppImage. I am not a fan of those package formats (AppImage, FlatPak, Snap) that basically put half an operating system into a package (at least all involved shared libs), even less so for QDirStat that is all about finding out where disk space got wasted, and then forcing the user to download and install 110+ MB to start the program.

But okay, if somebody got through the trouble of doing that, and if there are users out there who (for whatever reason) like that package format, it's their choice.

AppImage Pros and Cons

AppImage Pros

  • Independent of what Linux distribution you use: They say "Linux applications that run everywhere" (https://appimage.org/).

    Okay, let's give them the benefit of doubt.

  • You don't need to install dependent shared libs (the Qt libs in the case of QDirStat) because they are part of the package.

    The same could be achieved by good old static linking which however would be a violation of the LGPL under which many libraries are licensed, in particular the Open Source variant of Qt: The LGPL wants the end user to be able to exchange those shared libs with preferred versions that might contain custom modifications or security fixes.

AppImage Cons

  • Very large packages: Large download size, large installed size. This QDirStat AppImage weighs in at 113 MB, and that's already a compressed format: Trying to compress it with bzip2 results in the exact same size.

  • You cannot exchange the shared libs. If one of them gets a security update, you have to trust the AppImage packagers to provide an updated AppImage with the security fix. Worse, you don't even know if the AppImage has that fix. You can only hope that the AppImage packagers stay on top of all those security fixes that keep coming all the time, and that they distribute them without delay.

What Version?

Judging from the screenshot, this may not be the latest released version; the screenshot still has the old menu structure and no "Discover" toplevel menu.

That page https://apprepo.de/appimage/qdirstat doesn't say what version they packaged which is the first thing that greatly pisses me off. It says "build history: latest, 21-06-2021" which would be today.

But what does that mean? Today's QDirStat git version? Or the latest stable release built today with all the dependent libs (Qt, libz, GLibc) in their latest version? It doesn't tell.

That "SHA-1 Hash: 61e73908094d8c0ca47939db828140f74ad81be0" does not exist in the QDirStat GitHub repo:

sh@balrog:~/src/qdirstat$ git show 61e73908094d8c0ca47939db828140f74ad81be0
fatal: bad object 61e73908094d8c0ca47939db828140f74ad81be0
sh@balrog:~/src/qdirstat$ git log
commit 7ed60f20deb53b87e404860b814fd0d932945a63 (HEAD -> master, huha/master, huha/HEAD)
Author: Stefan Hundhammer <[email protected]>
Date:   Sun May 30 03:06:40 2021 +0200

    Don't kill performance with a crazy default key action

commit e7150bd56ca61a66a8c5a1630c4df39681fb27bb
Author: Stefan Hundhammer <[email protected]>
Date:   Wed May 19 17:56:11 2021 +0200

    Fixed warning about unchecked return value

commit 138ed33523e9cfa84fd9d30da4dbb95aafce05f9
Author: Stefan Hundhammer <[email protected]>
Date:   Wed May 19 17:46:30 2021 +0200

    Let the compiler optimize by default (issue #160)

commit 0b75489657312370db377d4b90f27b8c7a858a0d
Author: Stefan Hundhammer <[email protected]>
Date:   Fri Apr 30 00:18:29 2021 +0200

    Removed reference to outdated PPA

commit 1cf6b2e6526e187c554ec36079032393e5870801
Author: Stefan Hundhammer <[email protected]>
Date:   Mon Apr 5 17:20:49 2021 +0200

    Post-release version bump for devel versions

commit 735747a7d48324fc287ac5afb5b002e23e0c315c (tag: 1.7.1, huha/stable-1.7.1, stable-1.7.1, release-1.7.1)
Author: Stefan Hundhammer <[email protected]>
Date:   Mon Apr 5 15:30:12 2021 +0200

    Updated screenshot

commit 70a9f1276fa4b862af6f8dd3f152c0d9c5b003db
Author: Stefan Hundhammer <[email protected]>
Date:   Mon Apr 5 15:02:45 2021 +0200

    Change log

Okay, so let's download that thing and try to look inside.

Downloading and Starting the AppImage

After downloading:

sh@balrog:~/tmp$ ls -lh
total 113M
-rwxr-xr-x 1 sh sh 113M Jun 21 11:31 QDirStat.AppImage

Yikes. Compared with the freshly compiled binary:

sh@balrog:~/src/qdirstat/src$ ls -lh qdirstat
-rwxr-xr-x 1 sh sh 2.1M Jun 21 11:38 qdirstat

Which admittedly uses quite a number of shared libs (mostly the Qt libs and what they in turn require). But chances are that you have the Qt libs installed anyway for any of those other very common Qt applications, so the footprint of QDirStat is typically in the order of 2 MB, not 113 MB.

So let's start that AppImage:

sh@balrog:~/tmp$ ./QDirStat.AppImage 
/tmp/.mount_QDirStTb8US4/bin/qdirstat: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by /tmp/.mount_QDirStTb8US4/bin/qdirstat)
/tmp/.mount_QDirStTb8US4/bin/qdirstat: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by /tmp/.mount_QDirStTb8US4/lib64/libQt5Widgets.so.5)
/tmp/.mount_QDirStTb8US4/bin/qdirstat: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by /tmp/.mount_QDirStTb8US4/lib64/libQt5Gui.so.5)
/tmp/.mount_QDirStTb8US4/bin/qdirstat: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by /tmp/.mount_QDirStTb8US4/lib64/libQt5Core.so.5)
/tmp/.mount_QDirStTb8US4/bin/qdirstat: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /tmp/.mount_QDirStTb8US4/lib64/libQt5Core.so.5)
/tmp/.mount_QDirStTb8US4/bin/qdirstat: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by /tmp/.mount_QDirStTb8US4/lib64/libpng16.so.16)
/tmp/.mount_QDirStTb8US4/bin/qdirstat: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by /tmp/.mount_QDirStTb8US4/lib64/libicui18n.so.66)
/tmp/.mount_QDirStTb8US4/bin/qdirstat: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by /tmp/.mount_QDirStTb8US4/lib64/libicuuc.so.66)
/tmp/.mount_QDirStTb8US4/bin/qdirstat: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /tmp/.mount_QDirStTb8US4/lib64/libglib-2.0.so.0)

You gotta be kidding me.

The very purpose and raison d'être for the AppImage format is to avoid exactly this situation.
"Linux applications that run everywhere" was the promise!
And yet it failed to deliver with the most fundamental of all libraries, the GLibc?!? WTF?

Yes, it said on that download page:

It works in:

  • openSUSE Tumbleweed
  • Ubuntu 20.04
    ...
  • Linux distro with GLIBC >= 2.31

Does not work in:

  • Ubuntu < 20.04
  • Linux distro with GLIBC < 2.31

I have Ubuntu 18.04 LTS.

So what's the point of this AppImage format if it depends on the GLibc version that much?

If you violated the LGPL already for the Qt libs and all the others that you packaged, what prevents you from at least being consistent and remove this hard dependency on a specific GLibc version by simply packaging the GLibc as well?

Trying to Look Inside Manually

There must be a way to mount that thing. Let's see how this is supposed to work. A web search for "look inside appimage" gets us this as the first answer:

https://askubuntu.com/questions/1231597/how-can-i-examine-the-files-inside-an-appimage

How can I examine the files inside an Appimage?

From what I've read, an Appimage file is a compressed application along with all of its resources,
and when run it is auto-mounted and then executed.

I want to inspect the resources and files inside an Appimage file I've downloaded,
without actually running the Appimage.

How can I do this?

Answer

As such, an appimage can be mounted or extracted. That is:

my.AppImage --appimage-mount

...
mount -o loop

Okay, let's give that a try:

sh@balrog:~/tmp$ ./QDirStat.AppImage --appimage-mount
/tmp/.mount_QDirStQC67Ps

In a new shell window (because this one is now busy keeping that image mounted):

sh@balrog:/tmp/.mount_QDirStQC67Ps$ du -hs .
586M	.

Yikes. Unpacking that AppImage takes 586 MB!

sh@balrog:/tmp/.mount_QDirStQC67Ps$ ls
AppRun	bin  include  lib  lib64  libexec  QDirStat.desktop  qdirstat.svg  share

sh@balrog:/tmp/.mount_QDirStQC67Ps$ find . -name qdirstat -type f
./bin/qdirstat
./share/lintian/overrides/qdirstat

sh@balrog:/tmp/.mount_QDirStQC67Ps$ strings bin/qdirstat | grep 'QDirStat 1\.'
<h2>QDirStat 1.6.1</h2>

Adding insult to injury, they packaged QDirStat 1.6.1 which was released on 2020-02-13, i.e. 16 28 months ago. They didn't get around to package V 1.7 from 2020-07-26 (almost two years ago), V 1.7.1 from 2021-04-05, i.e. over one year ago, let alone V 1.8 from 2021-08-28.

Bottom Line

A package that does not tell what version you get and tends to get you outdated versions from well over a year ago is completely worthless.

Admittedly, that's not the fault of the package format, but of the packagers.

But it gets worse because the libraries they packaged in that package are just as outdated; it's still Qt 5.12, not Qt 5.15 which is the latest Qt 5.x version at the time of this writing.

If they can't get even those basics right, why should anybody trust them to ship libraries with the latest fixes, some of which are usually security fixes?

That, my friends, are reasons why I don't like those "shrink-wrap the world" package formats. Not AppImage, not FlatPak, not Snap.

@shundhammer
Copy link
Owner Author

shundhammer commented Jun 21, 2021

Filesystem Type of the Image

sh@balrog:/tmp/.mount_QDirStQC67Ps$ grep /tmp/.mount_QDirStQC67Ps /proc/mounts
QDirStat.AppImage /tmp/.mount_QDirStQC67Ps fuse.QDirStat.AppImage ro,nosuid,nodev,relatime,user_id=1000,group_id=1000 0 0

sh@balrog:/tmp/.mount_QDirStQC67Ps$ df -T .
Filesystem        Type                   1K-blocks  Used Available Use% Mounted on
QDirStat.AppImage fuse.QDirStat.AppImage         0     0         0    - /tmp/.mount_QDirStQC67Ps

Filesystem type fuse.QDirStat.AppImage ?!? Not very helpful.

Trying to mount the image manually after terminating that ./QDirStat.AppImage --appimage-mount with Ctrl C:

sh@balrog:~/tmp$ ls -lh
total 113M
-rwxr-xr-x 1 sh sh 113M Jun 21 11:31 QDirStat.AppImage

sh@balrog:~/tmp$ sudo mount -o loop ./QDirStat.AppImage /mnt
mount: /mnt: wrong fs type, bad option, bad superblock on /dev/loop0, missing codepage or helper program, or other error.

sh@balrog:~/tmp$ sudo mount -o loop -t fuse.QDirStat.AppImage ./QDirStat.AppImage /mnt
mount: /mnt: wrong fs type, bad option, bad superblock on /dev/loop0, missing codepage or helper program, or other error.

:-((

Is there even any different way than putting your trust in that very image that you downloaded and executing it? You can only hope that starting it with --appimage-mount does not execute malicious code.

That's nowhere near as secure as mounting it yourself with your system's mount command and a known filesystem type that is shipped together with your OS. I can be pretty sure that my system's mount command does not execute any code from inside the image that I am mounting.

@shundhammer
Copy link
Owner Author

Feel free to comment and discuss this here.

Since this is not a issue of QDirStat, closing for now.

@probonopd
Copy link

Hello, AppImage author here. Agree that AppImages should always come from the trusted original application author (you) and need to me made so that they run on all still-supported Linux distribution releases.

Would you be interested in an official AppImage that is actually working properly?

@shundhammer
Copy link
Owner Author

shundhammer commented Sep 14, 2021

If there is an AppImage out there, it should be up to date, it should always be clear what program version it is, and it should be as well-maintained as any distribution package; meaning that at least any critical security updates for libs that it uses are always kept up to date, and the versions of those libs should also be clear.

Those are the minimum requirements.

Having said that, this does not change any of my other points:

  • Size of the AppImage
  • Compatibility to distros that are reasonably old (at least the ones that are still supported)
  • Compliance to the GPL under which the libraries that are used are licensed, i.e. giving an end user a chance to exchange any of them.
  • Being able to look inside the package with standard tools, not having to put your faith into the very thing that you want to inspect by just starting it (!). This is an absolute no-no.

And in case this is unclear: I do not intend to build or distribute any such format, be it AppImage, FlatPak, Snap or similar.

I do build and distribute RPMs for SUSE Linux distributions only because I work for that company, I have easy access to the infrastructure to do it, and I have the know-how for it.

I don't build and distribute any other package format, neither Debian / Ubuntu .deb packages nor any other. For one thing, I don't have enough of the know-how (which could be fixed) nor the infrastructure nor the time.

Building binary packages is beyond the scope of an upstream Open Source author; this is the distribution maintainers' role. Not only is it time-consuming, there is also a lot more to maintaining a package than just throwing the source code and some description file (.spec or Debian rules file) into some build system and hope for the best. There are also distribution-specific rules and guide lines to follow, and, more importantly, keeping on top of security problems and their fixes.

This is even a lot worse for libraries, even more so for powerful and complex libraries like Qt; Qt is an all-encompassing framework that touches every corner, nook and cranny of programming, not only GUI, but also general programming like file handling, networking and whatnot. Somebody maintaining this needs extensive know-how of all of that, and, more importantly, needs to stay even more on top of security issues than a maintainer of a simple user program. Maintaining something like Qt and keeping it secure can be a tough job; this is certainly not something to do casually.

And that is actually my major gripe with all of those package formats that just include all dependencies in one package: I don't see how this can reasonably be done.

@probonopd
Copy link

probonopd commented Sep 14, 2021

If there is an AppImage out there, it should be up to date

Agree. I recommend building AppImages as part of an application's CI pipeline

it should always be clear what program version it is

Agree. The recommended naming is AppName-version-architecture.AppImage

it should be as well-maintained as any distribution package

Agree. In fact, it should imho be always the latest application version, not just the version any particular distribution happens to carry at that point.

Size of the AppImage

Will always be larger than distribution packages by the necessity of bundling everything privately that cannot reasonably be assumed to be part of all still-supported mainstream Linux distributions. This is an unavoidable tradeoff if you want a portable application that can run on all still-supported mainstream Linux distributions. But it may not as bad as one might think, since e.g., not the whole Qt needs to be shipped but only the parts that the application actually needs. And everything stays compressed on disk all the time, so the net effect depends a lot on the application -- in fact, e.g., LibreOffice is much smaller(!) in an AppImage than when traditionally installed on disk.

Compatibility to distros that are reasonably old (at least the ones that are still supported)

Agree. In fact, we reject any AppImage that won't run on the oldest still-supported Ubuntu LTS release from being included in https://appimage.github.io/.

Compliance to the GPL under which the libraries that are used are licensed, i.e. giving an end user a chance to exchange any of them

Agree. You can extract every AppImage and exchange any of its contents. Like you can with a zip or other archive file.

Being able to look inside the package with standard tools

Agree. I wish tools like Ark and other archiving tools would build in native AppImage support. Since an AppImage is basically just an ISO9660 or squashfs file with an ELF prepended to it, that should be no problem to do.

In the meantime, you can do it by:

  • Determining the size of the ELF file in bytes
  • Removing that number of bytes from the beginning of the AppImage file (or, more pracmatically, copy the rest of the file without those first bytes) to a new file
  • Extract the resulting ISO9660 or squashfs file with the usual tools
    You see, the format is not evil nor complicated.

I do not intend to build or distribute any such format

Sorry to hear that, but I respect your opinion, as you are the author.

I do build and distribute RPMs for SUSE Linux distributions only because I work for that company

For the sake of completeness, https://openbuildservice.org/ can build AppImages thanks to @adrianschroeter 👍

Building binary packages is beyond the scope of an upstream Open Source author; this is the distribution maintainers' role. (...)

Hence, OBS will do that work for you using all the well established distro-grade infrastructure. It will even re-build the AppImage automatically if one of the ingredients are updated on OBS.

So, other from the size argument (which is inherent to the objective of this format) I think all of your pain points are addressed?

@shundhammer
Copy link
Owner Author

If there is an AppImage out there, it should be up to date

Agree. I recommend building AppImages as part of an application's CI pipeline

...which would dump that whole problem into my lap again. No. The upstream author delivers the source code; that's it.

I know from my day job that those CI tools all have their quirks and problems, and they all need regular babysitting. There is not a single one of them out there that does not. I find myself dealing more with that stuff than with real development, and the benefits are questionable at best.

@probonopd
Copy link

The upstream author delivers the source code; that's it.

Disagree. (I mean, you are the author, you decide for your programs; but for the software I use I'd prefer to get it from an author who builds and supports it.)

Stallman says: Binaries without source code are worthless.
I say: Source code without (tested, supported) binaries (by the author) is equally worthless.
Not everyone needs to agree with me on this one, but it's the end users' perspective.

The sad reality is that some people (especially those in large organizations like universities) are forced to locked down (usually outdated) operating system releases and cannot "just install Tumbleweed" or whatever. Hence AppImage.

@shundhammer
Copy link
Owner Author

Size of the AppImage

Will always be larger than distribution packages by the necessity of bundling everything privately that cannot reasonably be assumed to be part of all still-supported mainstream Linux distributions. This is an unavoidable tradeoff if you want a portable application that can run on all still-supported mainstream Linux distributions. But it may not as bad as one might think, since e.g., not the whole Qt needs to be shipped but only the parts that the application actually needs. And everything stays compressed on disk all the time, so the net effect depends a lot on the application -- in fact, e.g., LibreOffice is much smaller(!) in an AppImage than when traditionally installed on disk.

LibreOffice is beyond the point here; it will probably benefit much from the compression that an AppImage uses to avoid completely insane package sizes.

But look at the numbers that I quoted above: 110 MB for the compressed QDirStat AppImage, 586 MB unpacked size.

I still remember the time when that was the total size of a complete Linux installation. This is especially bad when it's about an application for the dedicated purpose of combating disk space waste; this leads the whole idea pretty much ad absurdum.

There is also no good reason to waste that much disk space for the alleged benefit of having one package for all distributions; not only did that not work out in this case (see the Glibc incompatibility problems), it's also completely pointless when it's easy to build the real package for the real target distro instead.

Also, this can only work for machines with the same architecture; the argument falls apart the moment you need it for, say, an Arm machine like a RasPi.

@shundhammer
Copy link
Owner Author

shundhammer commented Sep 14, 2021

The upstream author delivers the source code; that's it.

Disagree. (I mean, you are the author, you decide for your programs; but for the software I use I'd prefer to get it from an author who builds and supports it.)

Stallman says: Binaries without source code are worthless.
I say: Source code without (tested, supported) binaries (by the author) is equally worthless.

No. It's reasonably easy to build; that's not rocket science.

Not everyone needs to agree with me on this one, but it's the end users' perspective.

The sad reality is that some people (especially those in large organizations like universities) are forced to locked down (usually outdated) operating system releases and cannot "just install Tumbleweed" or whatever.

They are always free to build their own. Or simply use the binary package that is available for their (even older) distribution. For SUSE distros, I even provide backported RPMs that go back quite a while (but here we really are in the extra-luxury area).

Hence AppImage.

Wrong conclusion IMHO. You are entitled to your opinion, of course, but I don't share it.

@shundhammer
Copy link
Owner Author

BTW no hard feelings here. I appreciate the discussion, even if we agree that we disagree. ;-)

@lslezak
Copy link

lslezak commented Nov 8, 2021

I don't build and distribute any other package format, neither Debian / Ubuntu .deb packages nor any other. For one thing, I don't have enough of the know-how (which could be fixed) nor the infrastructure nor the time.

Actually OBS can also build DEB packages. In the past we even built the YaST packages for Debian to run in Travis which uses Ubuntu in VMs. After adding Docker support we do not need that anymore, but you can still find the Debian packaging at https://github.com/yast/travis_old/tree/master_old/packages in case you are interested in details...

@shundhammer
Copy link
Owner Author

Reported upstream:

area-of-dev/QDirStat.AppImage#1

@shundhammer
Copy link
Owner Author

shundhammer commented Jun 5, 2022

So, that AppImage is not built from sources in that GitHub repo, they simply take the already built binary from Ubuntu 20.04 and repack it into AppImage format.

And it appears to be incredibly hard (too hard to accomplish if I read their response correctly) to query QDirStat's version number from that .deb package.

I am at a loss for words.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants