Host sources to add

[T145]

👋 Welcome!

This discussion is used to share and discuss implemented sources or sources that can be added in.

Reference sources that are currently implemented/reviewed here:

• https://github.com/T145/black-mirror#-sources=
• https://github.com/T145/black-mirror/blob/master/exports/sources.txt

Contributing Guidelines

Any discussions that actually have a discussion should be preserved: little information blurbs can be deleted after having been documented.

[iam-py-test]

@T145 can @DandelionSprout's antimalware list (here) be added to this blocklist?
Thanks!

[iam-py-test]

This seems interesting: https://github.com/DandelionSprout/adfilt/tree/master/Dandelion%20Sprout's%20Official%20DNS%20Server
A lot of the Dandelion lists are all over the place. I'm not sure if these are all manually maintained fully or not.

I am not sure if they are all maintained, however, his antimalware list is.
I have submitted issues & most are fixed within a day

[T145]

I really hate the fact that there's a garbage domain in this list, but I've added it in. I'd recommend blocking malc0de too, since its RSS feed contains Shade ransomware.

[DandelionSprout]

Those of my ~130 lists that were last updated more than 2 months ago, are those who've reached a state where there isn't all that much more that can be done to improve them, i.e. they're roughly as good as they can be at the moment.

The logistical intention of if-a-large-hosts-file-contains-this-entry-then-it.uses-dandelion-sprouts-anti-malware-list-as-a-source-list.data, was to make it possible to see which hosts-file compilation projects that used my lists, especially ones where source creditation was hard to find. However, since most of them (esp. 1Hosts and OISD) use various tools to remove dead domains from their lists, my plan did not work as well as intended.

[T145]

@DandelionSprout I'd recommend removing such garbage domains along that exact reasoning. I've seen some other lists do the same thing, and even repeat the same garbage host over and over in the same list. I'll be adding some optional host scanning on sources that aren't oriented towards blocking malware (since inactive hosts don't necessarily mean dead in that context), so it's not a big deal, but still unnecessary.

Thanks for your work!

[iam-py-test]

I really hate the fact that there's a garbage domain in this list, but I've added it in. I'd recommend blocking malc0de too, since its RSS feed contains Shade ransomware.

Malc0de is a malware-analysis website; I would not recommend blocking it

[iam-py-test]

@T145 please remove Anti-social-engineer as that list is unmaintained and has tons of False Positives
See TheAntiSocialEngineer/AntiSocial-BlockList-UK-Community#5 and TheAntiSocialEngineer/AntiSocial-BlockList-UK-Community#3

[T145]

Any of the popular "false positives" should be whitelisted by one of the whitelists I reference. I'll keep a finger on its pulse but it doesn't seem to be fully redundant. I'm planning to create a script that downloads each source and determines redundancy at some point, but this project is on the back-burner since I've started full-time work again.

[iam-py-test]

Please consider adding some of my lists (https://github.com/iam-py-test/my_filters_001) as sources
My main list is my antimalware list, which I update almost daily, but I have others for privacy, annoyances, ads, etc
I don't know if any of them meet your criteria, and if they don't, I can make changes
If they look ok, I can try and provide help with adding it as an source

[T145]

If you can identify any unique sources in your lists then I'd be happy to!

[iam-py-test]

If you can identify an unique sources in your lists then I'd be happy to!

Yay!
Some of my lists which you might want to include (hosts formats):

• Norton tracking protection list - provides protection against some of the tracking done by Norton and Avast (which is owned by Norton) (Note: This list does not cover Norton's main AV due to me being unable to get their installer without going through 20 steps and giving my credit card details)
• The malicious website blocklist - blocks scam sites, phishing, PUPs, and malware
• My anti-typo list - blocks possible typosquatt domains but is a bit aggressive
• My pornography blocklist - speaks for its self. Not updated much but technically supported (I would fix an issue if someone reported it)

All my lists are CC0 if that matters...

[T145]

I'm fairly certain the Norton and porn lists are fully covered. The Typo list is honestly not that aggressive, and I'd probably use ShadowWhisperer's typo list if blocking typos were important.

As for the malware list, it seems like there's definitely some overlap with other sources, and other typo domains that should belong in the typo list. The source majority seems to be taken from other sources too, so I'd add those instead to get constant updates.

[iam-py-test]

I'm fairly certain the Norton and porn lists are fully covered. The Typo list is honestly not that aggressive, and I'd probably use ShadowWhisperer's typo list if blocking typos were important.

As for the malware list, it seems like there's definitely some overlap with other sources, and other typo domains that should belong in the typo list. The source majority seems to be taken from other sources too, so I'd add those instead to get constant updates.

Ok.
Its really just your choice; this just shows I have room for improvement.

[iam-py-test]

Possible new HOSTS sources:
https://gitlab.com/curben/phishing-filter
https://gitlab.com/curben/pup-filter

[T145]

Phishing Filter:
This list takes openphish and phishtank data feeds and attempts to remove false positives by cross-referencing them with a Tranco, Umbrella, and OISD Exclusions subset. It's interesting due to the phishtank feed, but its integrity is compromised with the other operations. I'd pass on this one.

PUP filter:
This repository just provides alternate formats for the source material. I'd rather add malware-discoverer, especially since processing it is super simple with miller.

[iam-py-test]

Ok.
I'm not sure about the license of malware-discoverer though as I couldn't find it
Is there a procedure for finding new sources? Do they have to meet certain criteria?
I want to help, but don't want to waste your time with redundant sources

[T145]

All GitHub repositories are assumed All Rights Reserved unless a license is provided. Everything I do here is open, so I doubt anyone will have any qualms.

If you want to help, reference the contributing guidelines: https://github.com/T145/black-mirror/blob/master/CONTRIBUTING.md
And source listings: https://github.com/T145/black-mirror/tree/master/sources

The TODO list is a source list I gathered by cross-referencing implemented sources with several other popular blacklist projects, like Energized and 1Hosts. It would be a good place to start.

[iam-py-test]

Ok. Thank you

[T145]

Migrated from issues:

• https://www.misp-project.org/feeds/
• https://blocklist-tools.developerdan.com/blocklists

Requires importing

• https://www.circl.lu/doc/misp/feed-osint/

Generic

IPv4/IPv6

• https://home.nuug.no/~peter/pop3gropers.txt

[T145]

May be worth investigating: https://gitlab.com/quidsup/notrack/-/blob/master/src/blocklists.py

[T145]

Only good one: @hufilter https://github.com/hufilter/hufilter-dev

[T145]

May be worth investigating: https://pastebin.com/UQK9BxMu

[iam-py-test]

Oh. Thanks @nickspaargaren
Irrelevant question: do you mind if I copy part of your list into mine? (giving credit of course)

[nickspaargaren]

Of course and i would love it if you mention us! Just to keep in mind; we update the list regularly :)

[iam-py-test]

Of course and i would love it if you mention us! Just to keep in mind; we update the list regularly :)

Thank you!

[T145]

@T145 it seem the NoGoogle list entirely blocks Google, which could break a lot of things (including locking you out of any ReCAPTCA protected website)

I'm aware, and only plan on blocking Google trackers and advertisers. Energized Whitelist should whitelist Google services anyway.

[T145]

Found:

• @bjornstar: https://github.com/bjornstar/hosts
• @xinggsf: https://github.com/xinggsf/Adblock-Plus-Rule
• https://gitlab.com/Sorrow-San/9anime-adblock-filters/
• @hoshsadiq: https://github.com/hoshsadiq/adblock-nocoin-list
• @RPiList: https://github.com/RPiList/specials
  • Only unique & modern source from @Sainan / @hell-sh: https://github.com/hell-sh/Evil-Domains
  • https://hell.sh/hosts/sources.txt
  • Actually references some third-party sources: need to determine which are first-party.
• @LanikSJ: https://github.com/LanikSJ/ubo-filters
• @lassekongo83: https://github.com/lassekongo83/Frellwits-filter-lists
• @tofukko: https://github.com/tofukko/filter
• @unchartedsky: https://github.com/unchartedsky/adguard-kr
• @MetaMask: https://github.com/MetaMask/eth-phishing-detect/tree/master/src
• @cjx82630: https://github.com/cjx82630/cjxlist
• @ABPindo: https://github.com/ABPindo/indonesianadblockrules
• @kargig: https://github.com/kargig/greek-adblockplus-filter
• @gioxx: https://github.com/gioxx/xfiles

Calling it for now lol

[T145]

Interesting feeds referenced by https://www.covert.io/threat-intelligence/

[T145]

Found:

• @bongochong: https://github.com/bongochong/CombinedPrivacyBlockLists
  • WindowsTelemetryBlockSupplements seem original: will have to check if CrazyMax sources are made redundant.
  • @piperun: https://github.com/piperun/iploggerfilter

[bongochong]

Just chiming in to let you know that the list of mine in question is somewhat larger than CrazyMax's standard anti-windows-telemetry list, though it does include all of it, except for entries which have been reported to interfere with Windows Updates or the Microsoft Store.

[T145]

Great, thanks for confirming.

[iam-py-test]

@T145 I may be confused, but I think I have used the IPLoggerFilter before. It worked ok and I saw no False Positives

[iam-py-test]

You may want to consider removing Spam404 as it seems to be unmaintained, and domain reports to their website never seem to get added. uBlock Origin has already removed it and AdGuard seems to be considering it
See Spam404/lists#18 and uBlockOrigin/uBlock-issues#1766 (comment)

[T145]

I'm actually not using Spam404, but if you can find any used source that does let me know.

[iam-py-test]

Oh. I though I saw it in the list but either it was already removed or I am just confused
Maybe I was thinking of another repo?

[T145]

Another repo 🌴

[iam-py-test]

Another repo 🌴

Ok. Sorry.
You still could add it to the dead but mention-able section

[T145]

Grab all links from https://us-cert.cisa.gov/ncas/alerts.xml that start with https://us-cert.cisa.gov/sites/default/files/publications/ and scan the resulting files for any domains and IPs.

There are some domains in the feed formatted like so: domain[.]tld

[T145]

https://raw.githubusercontent.com/hl2guide/All-in-One-Customized-Adblock-List/master/adfilters_urls.txt

Found:

• @eEIi0A5L: https://github.com/eEIi0A5L/adblock_filter
  (Entries before are recorded or have no blocklist applications)

[T145]

Can we give some help to @LittleCordines?

https://github.com/LittleCordines/pfsense-hosts-file

[LittleCordines]

Yes. But how.. What kind of help..

[T145]

Maintaining the lists of course

[T145]

I'd go ahead and delete the IP lists though.

[T145]

Discover unique sources from: https://github.com/collinbarrett/FilterLists/blob/main/services/Directory/data/FilterList.json

[T145]

Discover unique sources from: https://github.com/Lennolium/PrivacyShield/blob/main/IncludedLists.md

[T145]

Discover unique sources from: https://github.com/serverless-dns/blocklists/blob/main/blocklistConfig.json