this feature is exported as crawler

• grep - grep all file download links from a site
• grepDir - grep all file download links from a webservers index site
• screenshot - take a screenshot from a website

• fetch all posts from a wordpress site, using json api

• probe Urls and get all header data

• push results to a http remote server (elasticsearch preferred)