Crawlers are the feeding creatures of the PiSec eco-system.

We like to think our crawlers like bees: as the bees go searching for flowers, they collect nectar and pollen and take it to the hive, where honey is produced.

In a way similar, our crawlers collect the list of dangerous links from external providers. Some examples are Phishstats (https://phishstats.info) and Phishtank (https://phishtank.org), but many more exists. Once the links are collected, these are sent to the central server, Brain (https://github.com/Ringloop/PiSec-Brain), to feed the central data repository.

At the moment, we only implemented Phishstats crawler as a proof of concept for the whole architecture, but we have plans to increment the number of sources.

Obviously contributions are more than welcome, so feel free to contact us!