Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

爬虫统计数据不对 #1415

Open
glacierck opened this issue Oct 26, 2023 · 2 comments
Open

爬虫统计数据不对 #1415

glacierck opened this issue Oct 26, 2023 · 2 comments
Labels
bug Something isn't working results

Comments

@glacierck
Copy link

Bug 描述
1、当我删除了爬虫下的任务记录后,回到爬虫列表,爬虫列表里的任务统计数据还是原来的任务数据量。
2、当我开启数据去重功能后,查看爬虫数据条目列表的总数和爬虫列表里的任务统计数据的数据总数也对不上。统计数据的数据依然把被去重的数据给加进去了。

复现步骤
该 Bug 复现步骤如下

  1. 随机启动多次任务,然后删除任务
  2. 启动去重功能,然后启动任务重复爬取,实验去重功能
  3. 返回项目下的爬虫列表,查看“统计数据”,进入爬虫明细,查看对应的数据。

截屏
image

image

image

@glacierck glacierck added the bug Something isn't working label Oct 26, 2023
@glacierck
Copy link
Author

image
ling'w另外补充一下,为啥爬虫数据不能删除。。。我测试爬虫代码的异常数据不是很尴尬?改完代码重跑前不能删数据,那我在线改代码有什么用。。。

@tikazyq tikazyq added the results label Dec 4, 2023
@TDxhfeng
Copy link

TDxhfeng commented Jun 3, 2024

image ling'w另外补充一下,为啥爬虫数据不能删除。。。我测试爬虫代码的异常数据不是很尴尬?改完代码重跑前不能删数据,那我在线改代码有什么用。。。

自己连接数据库找到crawlab库下的results_xxx表进行删除就可以了

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working results
Projects
None yet
Development

No branches or pull requests

3 participants