Legitimate sites are filtered due to inclusion of all domains from shallalist

[obbrc]

Looks like shallalist categorizes the domains, and not all of them are blacklistted. For example it has domains for governments or educational institutes. However, when processed, blackweb doesn't distinguish them so valid sites are blacklisted too.

A solution for this to include a category list (perhaps user can overwrite), and after untar, all valid categories can be removed.

I haven't checked other packages yet, but they may have similar files in them.

[obbrc]

Thanks for creating this by the way.

[maravento]

We sorry. The Blackweb project has no categories. We use our own list of allowed domains which are excluded from Blackweb. If you want to exclude a domain from our Blackweb list, you can report it, we check and include it