Quitten / robots.py Public

Notifications You must be signed in to change notification settings
Fork 1
Star 9

robots.py is a script was designed to observe and analyze robots.txt of targeted website in order to ease information security experts work. developed by Barak Tawily.

9 stars 1 fork Branches Tags Activity

Name		Name	Last commit message	Last commit date
Latest commit History 7 Commits
README.md		README.md
robots.png		robots.png
robots.py		robots.py

Repository files navigation

robots.py is a script was designed to observe and analyze robots.txt of targeted website in order to ease information security experts work.

Usage example:

robots.py http:https://insertYourHostHere.com

TODO:

Make it multithreaded.

Author:

Barak Tawily, application security expert.