Scan for sensitive information in Github easily and effectively.
The project is based on golang with AdminLTE to build a management system to manage the Github search results. Github API is utilized to crawl the related results according to key words and some rules. It proves to be a proper way to detect the information related to your company.:rocket::rocket::rocket: For a detailed introduction, you can refer here.
- Support multi platform, including Gitlab, Github, Searchcode
- Support search keyword in Huawei app store
- flexible rules
- utilize gobuster to brute force subdomain
git clone https://github.com/madneal/gshark
go get ./...
go build main.go
# check the config
mv app-template.ini app.ini
# start web service
./main web
# start crawler
./main scan
The configuration can be set according to app-template.ini. You should rename it to app.ini to config the project.
HTTP_HOST = 127.0.0.1
HTTP_PORT = 8000
MAX_INDEXERS = 2
DEBUG_MODE = true
REPO_PATH = repos
MAX_Concurrency_REPOS = 5
; server酱配置口令
SCKEY =
; gobuster file path
gobuster_path =
; gobuster subdomain wordlist file path
subdomain_wordlist_file =
[database]
;support sqlite3, mysql, postgres
DB_TYPE = sqlite
HOST = 127.0.0.1
PORT = 3306
NAME = misec
USER = root
PASSWD =
SSL_MODE = disable
;the path to store the database file of sqlite3
PATH =
- Make sure you have installed dependencies, suggest to use go mod
- Make sure the app.ini in config folder, you can rename app-template.ini to app.ini
- Make sure that you have config and set database correctly
- Make sure that you have config policy properly
- Make sure that you have config corresponding tokens for github or gitlab
You should build the main.go file firstly with the command go build main.go.
USAGE:
main [global options] command [command options] [arguments...]
COMMANDS:
web Startup a web Service
scan Start to scan github leak info
help, h Show a list of commands or help for one command
GLOBAL OPTIONS:
--debug, -d Debug Mode
--host value, -H value web listen address (default: "0.0.0.0")
--port value, -p value web listen port (default: 8000)
--time value, -t value scan interval(second) (default: 900)
--help, -h show help
--version, -v print the version
If it's the first time to run, there are some initial works will be finished automatically.
To execute main scan, you need to add a Github token for crawl information in github. You can generate a token in tokens. Most access scopes are enough. For Gitlab search, remember to add token too.
Now support notification by server 酱. Set the config of SCKEY in app.ini file.
- Access web service 403 forbidden
Access to https://127.0.0.1/admin/login
- Default username and password
gshark/gshark
# github.com/mattn/go-sqlite3 exec: "gcc": executable file not found in %PATH%
mattn/go-sqlite3#435 (comment)
go get ./... connection error
It's suggested to enable goproxy(refer this article for golang upgrade):
go env -w GOPROXY=https://goproxy.cn,direct
go env -w GO111MODULE=on
If you would like to join wechat group, you can add my wechat mmadneal with the message gshark.