Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

weibo.com关键字搜索只显示50页 #257

Closed
guanguxiansheng opened this issue Nov 5, 2022 · 4 comments
Closed

weibo.com关键字搜索只显示50页 #257

guanguxiansheng opened this issue Nov 5, 2022 · 4 comments
Labels
enhancement

Comments

@guanguxiansheng
Copy link

作者大大，weibo.com关键字搜索，只能爬取前50页了，有办法突破吗？微博移动版好像没限制
@nghuyong
Copy link
Owner

nghuyong commented Nov 5, 2022

可以这样试试，按照小时粒度拆分query；这样一天就可以产生24个搜索结果，每个50页，就有1200页的结果了

@guanguxiansheng
Copy link
Author

可以这样试试，按照小时粒度拆分query；这样一天就可以产生24个搜索结果，每个50页，就有1200页的结果了
嗯但这样每个小时内，几乎都只能爬取前几分钟的搜索结果

@wxqssr
Copy link

wxqssr commented Nov 18, 2022

可以这样试试，按照小时粒度拆分query；这样一天就可以产生24个搜索结果，每个50页，就有1200页的结果了
嗯但这样每个小时内，几乎都只能爬取前几分钟的搜索结果

我也是只能爬取前几几分钟的搜索结果

@nghuyong nghuyong mentioned this issue Nov 19, 2022
Closed
@nghuyong
Copy link
Owner

目前已经支持按照热度排序了，所以虽然不能突破50页的限制，但是可以采集到相对热门的前50页结果

nghuyong added a commit that referenced this issue Nov 25, 2022
@nghuyong
#257
f4c2cba
@nghuyong nghuyong mentioned this issue Jan 12, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@nghuyong @wxqssr @guanguxiansheng