Tags auto generation

[drew2a]

This PR is a part of #6214 and it introduces automatic tags extraction from torrent titles.

The following rules have been added:

1. Tags in square brackets: title [tag1, tag2, tag3]
2. Tags in parentheses: title (tag1, tag2, tag3)
3. A tag in an extension: title.tag

The tags could be extracted for all new torrents and they could be extracted for existing torrents in the background.

The extraction procedure for all new tags is pretty straightforward, they are just processed at the moment when they adding to MDS.

The background extraction procedure is a bit more complicated. This extraction makes by processing a batch of items. Every 10 seconds a batch of 1000 items has been processed. In the case tag rules processor rich the upper bound of DB, it starts from the beginning, but with increased interval (20 seconds) and increased batch size (2000 items).

Why 1000 items and 10 seconds? It is not too heavy for CPU and with these values, 360k items will be processed within the hour.

To distinguish processed items, the tag_version column has been added to TorrentMetadata:

    class TorrentMetadata(db.MetadataNode):
        ...
        tag_version = orm.Required(int, default=0)

Therefore it could be used in the future when we will process multiple incoming rules (we should just increase this value by 1 for each new rule set).

[devos50]

Overall, great work 👍 . I left a few minor comments and some concerns regarding the tag processor.

[devos50]

Could this error be related to this PR (Windows tests)?

src.tribler-core.tribler_core.components.key.tests.test_key_component.test_get_private_key_filename (from pytest)

pyfuncitem = <Function test_get_private_key_filename>

    def pytest_pyfunc_call(pyfuncitem):  # type: ignore[no-untyped-def]
        """Run coroutines in an event loop instead of a normal function call."""
        fast = pyfuncitem.config.getoption("--aiohttp-fast")
        if asyncio.iscoroutinefunction(pyfuncitem.function):
            existing_loop = pyfuncitem.funcargs.get(
                "proactor_loop"
            ) or pyfuncitem.funcargs.get("loop", None)
            with _runtime_warning_context():
                with _passthrough_loop_context(existing_loop, fast=fast) as _loop:
                    testargs = {
                        arg: pyfuncitem.funcargs[arg]
                        for arg in pyfuncitem._fixtureinfo.argnames
                    }
>                   _loop.run_until_complete(pyfuncitem.obj(**testargs))

c:\users\tribler\appdata\local\programs\python\python38\lib\site-packages\aiohttp\pytest_plugin.py:186: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
c:\users\tribler\appdata\local\programs\python\python38\lib\contextlib.py:120: in __exit__
    next(self.gen)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    @contextlib.contextmanager
    def _runtime_warning_context():  # type: ignore[no-untyped-def]
        """Context manager which checks for RuntimeWarnings.
    
        This exists specifically to
        avoid "coroutine 'X' was never awaited" warnings being missed.
    
        If RuntimeWarnings occur in the context a RuntimeError is raised.
        """
        with warnings.catch_warnings(record=True) as _warnings:
            yield
            rw = [
                "{w.filename}:{w.lineno}:{w.message}".format(w=w)
                for w in _warnings
                if w.category == RuntimeWarning
            ]
            if rw:
>               raise RuntimeError(
                    "{} Runtime Warning{},\n{}".format(
                        len(rw), "" if len(rw) == 1 else "s", "\n".join(rw)
                    )
                )
E               RuntimeError: 1 Runtime Warning,
E               c:\users\tribler\appdata\local\programs\python\python38\lib\site-packages\aiohttp\test_utils.py:560:coroutine 'interval_runner' was never awaited

c:\users\tribler\appdata\local\programs\python\python38\lib\site-packages\aiohttp\pytest_plugin.py:143: RuntimeError

[drew2a]

@devos50

Could this error be related to this PR (Windows tests)?

I guess it could, but I'm not sure.
Anyway, I've rewritten the execution of the TagProcessor's tests (the possible cause of the error), so will see.

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information

[devos50]

🤔 maybe wait with merging until the PR pipeline can complete?

[kozlovsky]

retest this please

[kozlovsky]

retest this please

[drew2a]

retest this please

[drew2a]

retest this please