Blackweb es un proyecto que pretende recopilar la mayor cantidad de listas negras públicas de dominios (para bloquear porno, descargas, drogas, malware, spyware, trackers, bots, redes sociales, warez, venta de armas, etc), con el objeto de unificarlas y hacerlas compatibles con Squid (Tested in v3.5.x). Para lograrlo, realizamos una depuración de urls, para evitar duplicados, dominios inválidos (validación de ccTLD, ccSLD, sTLD, uTLD, gSLD, gTLD, etc), y un filtrado con listas blancas de dominios (falsos positivos, como google, hotmail, yahoo, etc), para obtener una mega ACL, optimizada para Squid, libre de "overlapping domains" (e.g: "ERROR: '.sub.example.com' is a subdomain of '.example.com'").
Blackweb is a project that aims to collect as many public domain blacklists (to block porn, downloads, drugs, malware, spyware, trackers, Bots, social networks, warez, arms sales, etc.), in order to unify them and make them compatible with Squid (Tested in v3.5.x ). To do this, we perform a debugging of urls, to avoid duplicates, invalid domains (validation, ccTLD, ccSLD, sTLD, uTLD, gSLD, gTLD, etc), and filter with white lists of domains (false positives such as google , hotmail, yahoo, etc.), to get a mega ACL, optimized for Squid, free of overlapping domains (eg: "ERROR: '.sub.example.com' is a subdomain of '.example.com'").
| ACL | Black Domains | txt size | tar.gz size |
|---|---|---|---|
| blackweb.txt | 3.867.256 | 87.7 MB | 19.8 MB |
git squid bash tar zip wget subversion python
git clone --depth=1 https://github.com/maravento/blackweb.git
La ACL blackweb.txt ya viene optimizada para Squid. Descárguela y descomprimala en la ruta de su preferencia / The ACL blackweb.txt is already optimized for Squid. Download it and unzip it in the path of your preference
wget -q -N https://raw.githubusercontent.com/maravento/blackweb/master/blackweb.tar.gz && cat blackweb.tar.gz* | tar xzf -
wget -q -N https://raw.githubusercontent.com/maravento/blackweb/master/checksum.md5
md5sum blackweb.txt | awk '{print $1}' && cat checksum.md5 | awk '{print $1}'
El script bwupdate.sh descarga las FUENTES, las depura, unifica y elimina los dominios superpuestos. Una vez terminado verifique los logs de Squid y si aún existe algún error deberá depurarlo manualmente de blackweb / The bwupdate.sh script downloads the SOURCES, purifies, unifies and removes overlapping domains. Once finished check the Squid logs and if there is still an error you must manually debug it from blackweb
wget -q -N https://raw.githubusercontent.com/maravento/blackweb/master/bwupdate/bwupdate.sh && sudo chmod +x bwupdate.sh && sudo ./bwupdate.sh
- Antes de utilizar bwupdate.sh debe activar la regla en Squid-Cache / You must activate the rule in Squid-Cache before using bwupdate.sh
- Para reducir el tamaño de blackweb (eliminando dominios inactivos/muertos/inválidos) puede utilizar la herramienta httpstatus / To reduce the size of blackweb (eliminating inactive/dead/invalid domains) can use httpstatus
- El proceso de actualización y depuración de blackweb puede tardar mucho tiempo y consumir muchos recursos de hardware y ancho de banda / The process of updating and debugging of blackweb can take a long time and consume many hardware resources and bandwidth
- El path por default de blackweb es /etc/acl. Puede cambiarlo por el directorio de su preferencia / The default path of blackweb is /etc/acl. You can change it by the directory of your preference
- bwupdate.sh no incluye cloud/sync (Mega, Dropbox, Pcloud, iCloud, etc), soporte remoto (Teamviewer, Anydesk, logmein, etc) o dominios relacionados con telemetría, excepto si ya vienen bloqueados desde las FUENTES. Para bloquearlos o excluirlos debe activar las líneas en el script (# JOIN LIST), aunque se recomenda agregarlas manualmente a blackweb para evitar conflictos o falsos positivos / bwupdate.sh does not include cloud/sync (Mega, Dropbox, Pcloud, iCloud, etc) or remote support (Teamviewer, Anydesk, logmein, etc), or domains related to telemetry, except if they are already blocked from the SOURCES. To block or exclude them you must activate the lines in the script (# JOIN LIST), although it is recommended to add them manually to blackweb to avoid conflicts or false positives
REGLA Squid-Cache / Squid-Cache RULE
Edite / Edit:
/etc/squid/squid.conf
Y agregue las siguientes líneas: / And add the following lines:
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
acl blackweb dstdomain -i "/path_to_acl/blackweb.txt"
http_access deny blackweb
Blackweb contiene millones de dominios bloqueados, por tanto: / Blackweb contains millions of blocked domains, so:
- Utilice la ACL whitedomains para excluir dominios falsos positivos (y repórtelo) u otros dominios que quiera excluir (ejemplo: accounts.youtube.com desde Feb 2014, Google utiliza el subdominio accounts.youtube.com para autenticar sus servicios) / Use the ACL whitedomains to exclude false-positive (and report it) domains or other domains that you want to exclude (e.g.: accounts.youtube.com since Feb 2014, Google uses the subdomain accounts.youtube.com to authenticate its services)
- Utilice la ACL blackdomains para agregar dominios no incluidos en Blackweb (ejemplo: .youtube.com .googlevideo.com, .ytimg.com, etc.) / Use the ACL blackdomains to add domains not included in Blackweb (e.g.: .youtube.com .googlevideo.com, .ytimg.com, etc)
acl whitedomains dstdomain -i "/path_to_acl/whitedomains.txt"
acl blackdomains dstdomain -i "/path_to_acl/blackdomains.txt"
acl blackweb dstdomain -i "/path_to_acl/blackweb.txt"
http_access allow whitedomains
http_access deny blackdomains
http_access deny blackweb
adblockplus malwaredomains_full
ABPindo indonesianadblockrules
anudeepND Blacklist (included: coinminer, adservers)
Capitole - Direction du Système d'Information (DSI)
cedia.org.ec (included: domains, immortal_domains)
chadmayfield (included: porn_all, porn top)
CHEF-KOCH BarbBlock-filter-list
Disconnect.me (included: simple_ad, simple_malvertising, simple_tracking)
dshield.org (included: Low, Medium, High)
firebog.net (included: AdguardDNS, Airelle-hrsk, Airelle-trc, BillStearns, Easylist, Easyprivacy, Kowabit, Prigent-Ads, Prigent-Malware, Prigent-Phishing, Shalla-mal, WaLLy3K)
hosts-file.net (included: ad_servers, emd, grm, hosts, psh)
Malware-domains (or Lehigh Malwaredomains)
Matomo-org referrer-spam-blacklist
mitchellkrogza (included: Badd-Boyz-Hosts, Hacked Malware Web Sites, Nginx Ultimate Bad Bot Blocker, The Big List of Hacked Malware Web Sites, Ultimate Hosts Blacklist)
Perflyst (included: android-tracking, SmartTV)
quidsup (included: notrack-blocklists, notrack-malware, trackers, qmalware)
Ransomware Abuse (included: CryptoWall, Locky, Domain Blocklist, Ransomware Abuse ,URL Blocklist ,TorrentLocker)
securemecca.net and hostsfile.org
squidblacklist.org (included: dg-ads, dg-malicious.acl)
StevenBlack (included: add.2o7Net, add.Risk, fakenews-gambling-porn-social, hosts, spam, uncheckyAds)
zerodot1 CoinBlockerLists (included: Host, host_browser, host_optional, list, list_browser, list_browser_UBO)
UrlBlacklist (Server Down since July 24, 2017)
Passwall SpamAssassin (Server Down since Dec 2016)
Central Repo Dead Domains (Last Update included in Inactive Domains)
ipv6-hosts (Partial)
O365IPAddresses (Partial)
University Domains and Names Data List
Parse Domains (modified by Maravento)
Agradecemos a todos aquellos que han contribuido a este proyecto. Los interesados pueden contribuir, enviándonos enlaces de nuevas listas, para ser incluidas en este proyecto / We thank all those who have contributed to this project. Those interested can contribute, sending us links of new lists, to be included in this project
Special thanks to: Jhonatan Sneider
BTC: 3M84UKpz8AwwPADiYGQjT9spPKCvbqm4Bc
© 2019 Maravento Studio
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.