We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Originally appeared on StackOverflow.
When SPLASH_URL setting is missing the http:https:// scheme, scrapy's error is not very helpful
SPLASH_URL
http:https://
SPLASH_URL = 'localhost:8050' (...) 2017-06-01 14:44:35 [scrapy.core.scraper] ERROR: Error downloading <GET http:https://example.com/ via http:https://example.com/> Traceback (most recent call last): File "/home/paul/.virtualenvs/scrapy-splash/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1301, in _inlineCallbacks result = g.send(result) File "/home/paul/.virtualenvs/scrapy-splash/local/lib/python2.7/site-packages/scrapy/core/downloader/middleware.py", line 37, in process_request response = yield method(request=request, spider=spider) File "/home/paul/.virtualenvs/scrapy-splash/local/lib/python2.7/site-packages/scrapy_splash/middleware.py", line 358, in process_request priority=request.priority + self.rescheduling_priority_adjust File "/home/paul/.virtualenvs/scrapy-splash/local/lib/python2.7/site-packages/scrapy/http/request/__init__.py", line 94, in replace return cls(*args, **kwargs) File "/home/paul/.virtualenvs/scrapy-splash/local/lib/python2.7/site-packages/scrapy_splash/request.py", line 76, in __init__ **kwargs) File "/home/paul/.virtualenvs/scrapy-splash/local/lib/python2.7/site-packages/scrapy/http/request/__init__.py", line 25, in __init__ self._set_url(url) File "/home/paul/.virtualenvs/scrapy-splash/local/lib/python2.7/site-packages/scrapy/http/request/__init__.py", line 58, in _set_url raise ValueError('Missing scheme in request url: %s' % self._url) ValueError: Missing scheme in request url: render.html
Maybe the plugin can detect that the splash URL is wrong and fail with a more actionable message.
The text was updated successfully, but these errors were encountered:
No branches or pull requests
Originally appeared on StackOverflow.
When
SPLASH_URL
setting is missing thehttp:https://
scheme, scrapy's error is not very helpfulMaybe the plugin can detect that the splash URL is wrong and fail with a more actionable message.
The text was updated successfully, but these errors were encountered: