Scraping Boardgamepage Shops using Playwright and oclif.io Data can be filtered using scrappy-web
- Clone repository
- Enter the project directory and execute
npm install
in order to install all the packages
npm install playwright
npm install @playwright/test
./bin/dev <command> --save <flag>
to run with a flag to savesave
:json
, console.log() as default- use
--help
for help - debug with
DEBUG=* ./bin/dev command
npm run scrappy
to run an examplenpm run scrappy:parallel
to run all scrapers parallellynpm run <shopname>:json
to run single shop scraper, e.g.npm run dragoneye
- available shops:
dragoneye
,dragonus
,mepel
,planszostrefa
,shopgracz
- scraping only available games if possible