-
Notifications
You must be signed in to change notification settings - Fork 451
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
custom user-agent #109
Comments
Hey @cp2587, If you're using Lua scripts then headers are not handled automatically. From scrapy-splash readme (
Scroll to the last example in the https://github.com/scrapy-plugins/scrapy-splash#examples section - there is an example of how to make headers, cookies, etc. work with a Lua script. |
Does it work with Scrapy fake user agent? |
Hello,
We realized recently that altought we had set custom user-agent in headers params of splash/args when doing a request, this header was never used by splash afterward. For now the only working fix we found, is using 'splash:set_user_agent' directly into our lua scripts.
Are you aware of this bug ? Did we miss something ?
The text was updated successfully, but these errors were encountered: