[rt.cpan.org #132811] Win32: Crash a week after start

Fri Jun 12 09:37:57 2020: Request 132811 was acted upon.
Transaction: Ticket created by [email protected]
Queue: PAR-Packer
Subject: Win32: Crash a week after start
Broken in: (no value)
Severity: (no value)
Owner: Nobody
Requestors: [email protected]
Status: new
Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=132811 >


Hi,

I just found out why PAR::Packer-generated executables tend to crash when I interact with them after I started them some days ago and they stayed mostly idle in the task bar -- this also happened on Win7, but I analyzed it under Win10:

https://support.microsoft.com/en-us/help/4506040/temp-folder-with-logon-session-id-is-deleted-unexpectedly

You can imagine a program behaving strange or crashing when it wants to load modules, resources or data files from its PAR_TEMP directory, but everything but the already open DLLs has vanished. I don't know exactly which windows versions are affected, but I don't want to give programs to users that suddenly crash out of nowhere, they might lose trust into the programs.

I don't know if there can be a better solution than documenting this behaviour. Of course one can set PAR_GLOBAL_TEMP or PAR_GLOBAL_TMPDIR, but you have to use wrapper scripts or an installer that find a suitable directory on the user's machine and set the variables for the process or globally -- which a bit defeats the 'one executable, no installer, just double-click the file I gave you' purpose of PAR::Packer. So the workaround I found is (without the Win32API::File-doesn't-exist-on-non-Win32 handling):

if ('CODE' eq ref $INC[-1] && $^O eq 'MSWin32' && !$ENV{PAR_GLOBAL_TEMP} && !$ENV{PAR_GLOBAL_TMPDIR}) {
use File::Find;
use Win32API::File qw(:ALL);
#use threads;
#(async
{
no warnings 'File::Find';
find +{
no_chdir => 1,
wanted => sub {
# ignore result, handle stays open until program terminates
-f and createFile $_, 'r ke', 'rw';
},
}, $ENV{PAR_TEMP};
}
#)->detach;
}

Is there a better way?

Mit freundlichen Grüßen
Ralf

Fri Jun 12 19:44:29 2020: Request 132811 was acted upon.
Transaction: Correspondence added by SLAFFAN
This is related to https://rt.cpan.org/Public/Bug/Display.html?id=101800

Shawn.

Fri Jun 12 21:35:12 2020: Request 132811 was acted upon.
Transaction: Correspondence added by [email protected]
Queue: PAR-Packer
Subject: RE: [rt.cpan.org #132811] Win32: Crash a week after start
Status: open
Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=132811 >


Hi.
Yes and no. _CANARY_.txt exists and the directory will be repopulated on the next start of the program, so there is no problem except having to unpack the files again between runs of the program. My problem on the other hand is, that I start the program and the files are deleted while it is running. My crude protection scheme (locking all of the files -- 3500 files in my case, which only works with Windows file handles since Perl only gets the 2048 file handles implemented by the Windows libc) only protects the files while the program is running, so _CANARY_.txt is still needed (and there is a short time window at the start of the program, inbetween starting the unpacking and locking the files, where files could still vanish if you are unlucky enough to have cleanmgr.exe run at exactly the same time.

Using a non-temp-directory would solve both problems but carries some problems on its own, see original message. Declaring %TEMP% to be a non-temp directory sounds strange. Raising the existence interval only makes it less bad, but doesn't solve it. Also you have to have administrator privilege for both.

I also thought about keeping the files fresh by manipulating their timestamps, that also doesn't really solve it, but reflects the truth that the files have been used very recently. The would be easy but time-consuming on startup (and make _CANARY_.txt obsolete). To solve the runtime problems you would have to spawn a parallel job to keep the files fresh every couple of hours.

In essence, the reason is the same, but I don't see a simultaneous solution for both aspects that doesn't require user oder administrator input.

Mit freundlichen Grüßen

Ralf

Fri Jun 12 22:04:55 2020: Request 132811 was acted upon.
Transaction: Correspondence added by SLAFFAN
Queue: PAR-Packer
Subject: Win32: Crash a week after start
Yes, the canary file only helps in cases where files are deleted between program runs or where files only need to be read at startup.

The regular temp file cleanup will also impact all users in that the PAR archive needs to be unpacked each time it is cleaned up, causing a slow user experience for large archives at startup.

FWIW, I'd be OK if the default PAR_TEMP location was in the user's home dir or under C:\ProgramData ($ENV{PROGRAMDATA}). The latter might have issues on shared machines depending on how the unpacked files are used, though.

Shawn.

did you consider to use pp to crete the set of required files, but *not*
at runtime?

You can then put all the required files to a safe directory, optionally
even write protected for ordinary users to avoid unexpected modification.

I made a simple launcher replacing perl.exe if your users don't want to
need to invoke perl.exe yourapplication.pl

I remember there is even an "unpacker" from another user to extract all
the needed files from a pp created exe archive.

Oliver

Sat Jun 13 02:59:53 2020: Request 132811 was acted upon.
$ENV{APPDATA} would be better than $ENV{PROGRAMDATA}

It is easy enough to add to mktmpdir.c for Windows builds, so I can work up a PR if Roderich is in favour of the idea.

Sat Jun 13 06:44:03 2020: Request 132811 was acted upon.
Subject: RE: [rt.cpan.org #132811] Win32: Crash a week after start
Hi.
What is your concept for getting rid of old PAR_TEMP locations once you get (or create) an updated version of the application or just stop using it? Explicit deinstallation (maybe by using a special argument or variable when calling this version)? An own version of 'delete old files' but with a longer timeout and using a better protocol for detecting the last use and removing only whole directories? Giving the user better hints than cryptic directory names with cryptic files in it, that he can decide which dirs to purge manually? This could be implemented by description files or by a version manager.

Explicit deinstallation could be paired with explicit installation -- the application only unpacks intelf into a permanent location, if you explicely tell it to do so, in a way every user understands.

Mit freundlichen Grüßen

Ralf Neubauer

Sat Jun 13 18:45:02 2020: Request 132811 was acted upon.
Transaction: Correspondence added by SLAFFAN
Queue: PAR-Packer
Subject: Win32: Crash a week after start
Currently there is no removal of old PAR archives that I am aware of. Removal is left to the user or automated processes.

It could be done manually using PAR_GLOBAL_CLEAN at the cost of running the exe. e.g. for some_archive.exe this could be packed into an uninstall.bat:

set PAR_GLOBAL_CLEAN=1
some_archive.exe

Maybe this could be added to PAR so it does not run the exe and just does the cleanup, e.g.:

some_archive.exe --run-par-cleanup

It's probably conceptually simpler for the user if it is a separate process, though.

It would also be possible to add that logic to your own perl script so it can exit without doing anything, but it might be too late by the time that is loaded.

Sat Jun 13 23:32:46 2020: Request 132811 was acted upon.
I've run some testing and neither approach works. When PAR_GLOBAL_TEMP is set before running the exe, the archive is unpacked into a different folder named like temp-12345 and cleaned up on exit. Setting it in a BEGIN block within the packed script appears too late to have any effect. Presumably the environment is localised or some other flag is set by PAR at startup to check on exit.

It looks like a separate uninstaller is needed, although it would need to ensure it does not remove files from under a running process, as otherwise it is another example of the original reported issue.

Shawn.

Sun Jun 14 11:24:18 2020: Request 132811 was acted upon.
Transaction: Correspondence added by RSCHUPP
Go for it :) I suggest "$ENV{APPDATA}/Local/pp" and maybe we may omit the per-user sub directory
in this case (as %APPDATA% should already be per-user). Maybe drop the ancient fallback to C:\TEMP
(and the really weird %WinDir%\temp) as well.

Keep in mind that par_mktmpdir in myldr/mktmpdir.c has a perl-level sibling _set_par_temp in script/par.pl.

Cheers, Roderich

Sun Jun 14 11:49:03 2020: Request 132811 was acted upon.
There is none.
Nope. PAR::Packer is expressly designed to work WITHOUT installation and tacking on stuff
is out of the question - there are already too many special arguments/variables.
If you want installation/deinstallation look somewhere else.

Cheers, Roderich

Neubauer, Ralf wrote:

[PAR_TEMP]
My simple concept is not to hide the unpacking but tell the users the truth:

There is a bunch of files needed to run the Perl application, and if you
don't use it anymore, you need to remove it.

I decided to use the PAR packager only to collect the needed files but
not to use it's runtime magic.

You find an extraction tool at
https://github.com/ChordPro/chordpro/blob/dev/pp/pp2ppl.pl

My primitive launcher is here:
https://oliverbetz.de/pages/Artikel/Portable-Perl-Applications

If you want to provide an uninstaller, use Inno Setup, NSIS,
InstallShield or something similar for a clean solution.

Oliver

Sun Jun 14 16:21:45 2020: Request 132811 was acted upon.
Subject: RE: [rt.cpan.org #132811] Win32: Crash a week after start
Hi,
> There is none.

This question was meant to be an answer to Shawns ideas. He devised de-facto-installation and I wanted to hear the fully thought out story from him.
If the files are in %TEMP% someday someone will come along and clean up, because it is a known place for cleaning up potential. No one will look into Appdata/pp if the disk fills up.
Unpacking into something else than %TEMP% is de facto installation. I don't want installation, but if the archive de facto installs itself into a hidden permanent directory, there should be some help for the user to keep the space usage under control. Sometimes I create new versions every couple of days and if people run that, their disks will fill up. Completely ignoring the problem means the deinstallation process is to manually clean the correct ones of a big collections of directories with cryptic names from a not-so-well-know subdirectory of Appdata. I read most of the source of the package and still hope there can be some middle ground to make it a bit easier for the users. One of my suggestions was to at least describe the contents of the directories, for every cache-xyz/ there could be a cache-xyz-description.txt which lists the name of the application, the date of unpacking. And maybe the timestamp of cache-xyz/ and cache-xyz-description.txt should be set to the curr
ent time every time the application is started, that would make it possible to find the old unused entries with a sorted directory listing.

As an alternative to Shawns solution you could also unpack into %TEMP% like before, but set a timestamp some interval (e.g. 6 months) in the future for all files every time the application starts, that would most likely also protect the files.

Mit freundlichen Grüßen

Ralf