-
Notifications
You must be signed in to change notification settings - Fork 506
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
功能与意见反馈,报bug可以另开issue #6
Comments
感谢作者分享,简单试用了一下,有几个想法
谢谢 |
@jxltom 很不错的建议 ,谢谢 ,后续版本会改善这些问题。 |
@DormyMo 好的。我看到UI是用flask运行的,请问nodejs在项目里发挥什么作用? 对于Flask的话,如果实现Http auth,Flask-BasicAuth插件几行代码就可以实现http basic authentication,可以考虑一下 |
您好,请问SpiderKeeper可以在win下跑起来吗?需要scrapyd? |
请教一下spider的优先级是什么意思,具体是怎样实现的? |
@bosbyj SpiderKeeper 基于scrapyd服务,需要scrapyd,win下可以,不过没有测试过 |
@PythonYXY 现在优先级相当于 运行爬虫的机器数,优先级越高,会再越多的机器上运行同一个爬虫(分布式执行,需要scrapy-redis) |
@DormyMo 嗯,现在我就是正在做分布式爬虫。所以如果A的优先级比B高,就相当于A会比B多执行几次“scrapy crawl A”的命令么? |
试用了下,spiderkeeper 直接提出了一些 warning,可以顺手解决一下:
|
能不能加上监控简单脚本的功能啊 |
都可以在这里交流,我会及时回复的~
也欢迎加入QQ群讨论:389688974
The text was updated successfully, but these errors were encountered: