Skip to content

I'm trying to finish the scraplat as a scraper platform

License

Notifications You must be signed in to change notification settings

VillanCh/scraplat

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

24 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

SCRAPLAT a thread pool crawler platform

SCRAPLAT 一个使用线程池调度的爬虫平台

0x00 intro 简介

This proj make scarping web easier, scraplat is based on threading pool and the manager controls many scrapers to work.

You can use BeautifulSoup or selenium (even HTMLParser or SGMLParser)to create your own handler

如果你需要进行爬虫的爬取的话,这个项目简化了构建多线程爬虫步骤。这个项目多线程爬虫基于线程池实现线程调度,一个manager分配给控制多个worker工作

你可以使用BeautifulSoup或者是selenium(甚至HTMLParser或者是SGMLParser)来编写脚本

0x01 python版本与依赖 TODO

0x02 目标 TODO

0x03 TODO

About

I'm trying to finish the scraplat as a scraper platform

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages