Allow to store "resume data" in SQLite database file

[glassez]

It is available as "experimental" advanced option.
Existing "resume data" is transferring in selected storage once you start qBittorrent with changed option.
libtorrent native "resume data" is stored in bencoded form in single database column to simplify first implementation. Actually we don't need to split it into separate columns until we really implement some improvements that need such data layout. But since until then we can't have an accurate idea of what kind of layout and data format of individual columns we will need, it is completely useless to do any kind of separation now.

[thalieht]

When going from sqlite -> fastresume, shouldn't it create the queue file on startup like everything else in BT_folder?
Other than that i found no problems.

[glassez]

When going from sqlite -> fastresume, shouldn't it create the queue file on startup like everything else in BT_folder?

It was done initially in an optimistic/lazy manner (it just allowed the queue to be saved at its regular time, at least when the application was shut down). Now it should do this immediately after the storage conversion is done.

[Chocobo1]

it seems currently we don't need to be worried about sql injections right? All code paths only uses our own defined strings/columns.

[glassez]

it seems currently we don't need to be worried about sql injections right? All code paths only uses our own defined strings/columns.

I don't think there is any acute problem here, but we still need to evaluate it further. Perhaps some of the first users will be interested in researching it.
In any case, this "feature" is not intended for widespread use in the first stages.

[FranciscoPombal]

(this is kind of a non-review)

This seems really awesome but haven't had the time to test this out, so I only left some minor comments. Nevertheless, after those are addressed, I'll immediately approve this, because it is off by default and it is gated behind an advanced settings toggle labeled "experimental" - because of this, there isn't any risk in breaking things for those who are not prepared to risk that. It seems desireable to get this into the hands of a larger audience of power users more quickly for feedback than to sit on it for long.

Also, I don't have much experience with using such tools to store program data, so I have a few questions:

• How easy does this makes upgrading to a new format? IMO the biggest pain point in changing the fast resume code is having to introduce janky temporary "upgrade code". I understand this solves that particular problem, but I'd be interested in some details, if you could explain them please.
• Similarly, what happens if the user wants to downgrade? Can the database also automatically downgrade, or would it be backwards-incompatible? If the latter is the case, what would a user have to do to downgrade the database (I'm mainly interested in knowing whether or not such a task can be accomplished by an average user)?
  The current solution does not offer strong guarantees when downgrading between versions where the fastresume format changes; on the other hand, it "just works" if the format hasn't changed between versions.

[glassez]

How easy does this makes upgrading to a new format?

If you mean an upgrade from one database layout to another, then it looks at first glance comparable in complexity to other data storage formats. Much depends on the changes in the data itself that need to be stored.

I'd be interested in some details, if you could explain them please.

As I mentioned above, it depends on the specific case. To begin with, I added simple versioning so that we can later more easily determine the current data format to decide if it needs an upgrade.

As for the downgrade, we don't really care about it even now. For me, such concern generally looks counterproductive. We just don't have to make changes in places like this (data storage format, etc.) too often. The user must take care of the backup of the current data during a major update, in case he decides to downgrade later.
In fact, there is some elegant solution in the case of SQL database. When updating the database layout, we can put the downgrade logic in the form of a SQL code in the service table, and implement a downgrade support mechanism that will check the current version of the data format, and if it is upper than the supported one, look for the downgrade code in the database itself and execute it. However, I am sure that many people will reject this solution because of possible (or imaginary) security problems.

As for the first implementation, it will work almost the same as current bencoded (.fastresume) file based storage, since I just save all libtorrent resume data in a single table field, using separate fields only for qBittorrent own data items.

[zero77]

@glassez
Should this be implemented but, not set as the default straight away so that if there is a bug people can easily revert without this being an issue.

Then once there are a couple of stable releases that have this included change it to the default that way the previous stable version still supports it so reverting won't be a problem.

[glassez]

@zero77
It will be added as an "experimental" feature that is disabled by default (intended only for advanced users). And I intend to keep it that way for quite some time.

[FranciscoPombal]

Approved as per #14726 (review).

[glassez]

PR is rebased on top of current master to get compatible with new tags implementation. Need to be formally approved so I can merge it.

[sledgehammer999]

libtorrent native "resume data" is stored in bencoded form in single database column to simplify first implementation. Actually we don't need to split it into separate columns until we really implement some improvements that need such data layout.

I am glad that this was the direction like my own initial try. I couldn't then figure out your suggestion for separating out the bencoded data. And something to ponder for future iterations: Does it make sense for us to dissect the native libtorrent format, when that format can change in the future?

Sadly I was not able to follow through with that old PR.

[glassez]

I couldn't then figure out your suggestion for separating out the bencoded data.

These are memories from the era before libtorrent-1.2. But now we have resume data in the form of add_torrent_params structure. So there is no question of excessive transcoding right now. The only question is the format in which to store this data.

And something to ponder for future iterations: Does it make sense for us to dissect the native libtorrent format, when that format can change in the future?

It might make sense if we get some benefit from it, so the extra care will be justified.

[jagannatharjun]

gets this on the first startup after setting storage type to sqlite (ignore qbittorrent version it's built on top of the current master),

qBittorrent has crashed
Please file a bug report at http:https://bugs.qbittorrent.org and provide the following information:

qBittorrent version: v4.2.5 (64-bit)
Libtorrent version: 2.0.3.0
Qt version: 5.15.2
Boost version: 1.72.0
OpenSSL version: 1.1.1d
zlib version: 1.2.11
OS version: Windows 10 Version 2009 10.0.19042 x86_64

Caught signal: SIGABRT

#  0 qbittorrent.exe      0x00007ff7e96023f8 straceWin::getBacktrace()[ C:\Users\Prince\Desktop\Projects\qBittorrent\src\app\stacktrace_win.h : 220 ]
#  1 qbittorrent.exe      0x00007ff7e9602ff7 sigAbnormalHandler(signum)[ C:\Users\Prince\Desktop\Projects\qBittorrent\src\app\main.cpp : 367 ]
#  2 ucrtbase.dll         0x00007ffaf6161881 raise()
#  3 ucrtbase.dll         0x00007ffaf6162851 abort()
#  4 ucrtbase.dll         0x00007ffaf6161f9f terminate()
#  5 qbittorrent.exe      0x00007ff7e97c773a __scrt_unhandled_exception_filter(pointers)[ D:\a01\_work\9\s\src\vctools\crt\vcstartup\src\utility\utility_desktop.cpp : 93 ]
#  6 KERNELBASE.dll       0x00007ffaf5dbb7c7 UnhandledExceptionFilter()
#  7 ntdll.dll            0x00007ffaf8535170 memset()
#  8 ntdll.dll            0x00007ffaf851c776 _C_specific_handler()
#  9 ntdll.dll            0x00007ffaf853207f _chkstk()
# 10 ntdll.dll            0x00007ffaf84e1454 RtlRaiseException()
# 11 ntdll.dll            0x00007ffaf84e11a5 RtlRaiseException()
# 12 KERNELBASE.dll       0x00007ffaf5ce4b59 RaiseException()
# 13 VCRUNTIME140.dll     0x00007ffaef576480 CxxThrowException()
# 14 qbittorrent.exe      0x00007ff7e966fea6 BitTorrent::DBResumeDataStorage::DBResumeDataStorage(dbPath, parent)[ C:\Users\Prince\Desktop\Projects\qBittorrent\src\base\bittorrent\dbresumedatastorage.cpp : 188 ]
# 15 qbittorrent.exe      0x00007ff7e962c78d BitTorrent::Session::startUpTorrents()[ C:\Users\Prince\Desktop\Projects\qBittorrent\src\base\bittorrent\session.cpp : 4137 ]
# 16 qbittorrent.exe      0x00007ff7e95f94b5 Application::exec(params, params)[ C:\Users\Prince\Desktop\Projects\qBittorrent\src\app\application.cpp : 682 ]
# 17 qbittorrent.exe      0x00007ff7e96038a9 main(argc, argv, argv)[ C:\Users\Prince\Desktop\Projects\qBittorrent\src\app\main.cpp : 310 ]
# 18 qbittorrent.exe      0x00007ff7e97ca8d7 WinMain()
# 19 qbittorrent.exe      0x00007ff7e97c6fb2 __scrt_common_main_seh()[ D:\a01\_work\9\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl : 288 ]
# 20 KERNEL32.DLL         0x00007ffaf76f7034 BaseThreadInitThunk()
# 21 ntdll.dll            0x00007ffaf84e2651 RtlUserThreadStart()

[jagannatharjun]

gets this on the first startup after setting storage type to sqlite

my bad there was something wrong with my CMake configuration.

[xavier2k6]

@glassez Since Appveyor was updated to MSVC 2019 it consistently shows warning C4101: 'err': unreferenced local variable below:

qBittorrent/src/base/bittorrent/dbresumedatastorage.cpp

Line 389 in 365b1c6

catch (const RuntimeError &err)

qBittorrent/src/base/bittorrent/dbresumedatastorage.cpp

Line 566 in 365b1c6

catch (const RuntimeError &err)

Can this be ignored?

[glassez]

Can this be ignored?

Yes. But we still need to fix it. It's a regression of 05e3e46.
It should be catch (const RuntimeError &).

[xavier2k6]

we still need to fix it

Will you do it or will I?

PR title??
Fix throw exception handling (05e3e46 regression).

[Chocobo1]

Will you do it or will I?

I'll fix it in #14976.