Skip to content

pixelindigo/arachnarium

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Arachnarium

Manual installation

python -m pip install .

Usage

Run a single experiment with a web app arachnarium run CRAWLER_PATH APP_PATH CRAWLER_ARGS

Example: arachnarium run examples/crawlers/simplecrawler examples/apps/simplewebapp -t 1 http:https://web/

Run a single experiment with a website arachnarium run CRAWLER_PATH WEBSITE_URL CRAWLER_ARGS

Example: arachnarium run examples/crawlers/simplecrawler https://cispa.de -t 1 -n 5 https://cispa.de

Run a batch experiment with a website arachnarium run -w NUM_WORKERS BATCH_FILE

Example: arachnarium batch -w 4 examples/batch.yml

Then check experiments/ directory to see the generated data -- each experiment would be at /<app>/<crawler>/<id>/ and contain the following dir structure:

  • coverage/ -- code coverage files collected (if applicable)
  • report/ -- crawler's report files
  • command.txt -- crawler command line arguments
  • runtime.txt -- time spent on an experiment
  • stderr.txt -- stderr output
  • stdout.txt -- stdout output

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Packages

No packages published

Languages