-
Notifications
You must be signed in to change notification settings - Fork 639
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
scrapy运行无日志,gerapy请求则会报错 #263
Labels
Comments
这个问题是views.py中的job_log这个函数在发起请求时的请求头参数headers={'Range': 'bytes=-1000'}引起的,当日志信息为空时,发起请求获取最后1000字节的数据,会使scrapyd发生异常,这应该是scrapyd的问题 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2023-04-08T15:21:06+0000 [twisted.web.server.Request#critical]
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/twisted/web/server.py", line 292, in render
body = resrc.render(self)
File "/usr/local/lib/python3.11/site-packages/twisted/web/_template_util.py", line 194, in render
self.d.addCallback(self._cbChild, request).addErrback(self._ebChild, request)
File "/usr/local/lib/python3.11/site-packages/twisted/internet/defer.py", line 532, in addCallback
return self.addCallbacks(callback, callbackArgs=args, callbackKeywords=kwargs)
File "/usr/local/lib/python3.11/site-packages/twisted/internet/defer.py", line 512, in addCallbacks
self._runCallbacks()
--- ---
File "/usr/local/lib/python3.11/site-packages/twisted/internet/defer.py", line 892, in _runCallbacks
current.result = callback( # type: ignore[misc]
File "/usr/local/lib/python3.11/site-packages/twisted/web/_template_util.py", line 200, in _cbChild
request.render(resource.getChildForRequest(child, request))
File "/usr/local/lib/python3.11/site-packages/twisted/web/server.py", line 292, in render
body = resrc.render(self)
File "/usr/local/lib/python3.11/site-packages/twisted/web/_auth/wrapper.py", line 198, in render
return super().render(request)
File "/usr/local/lib/python3.11/site-packages/twisted/web/resource.py", line 268, in render
return m(request)
File "/usr/local/lib/python3.11/site-packages/twisted/web/static.py", line 657, in render_GET
producer.start()
File "/usr/local/lib/python3.11/site-packages/twisted/web/static.py", line 770, in start
self.fileObject.seek(self.offset)
builtins.OSError: [Errno 22] Invalid argument
2023-04-08T15:21:06+0000 [twisted.python.log#info] "172.17.0.1" - - [08/Apr/2023:15:21:05 +0000] "GET /logs/VulSpiders/cnnvdSpider/f7a99998d62011ed94c70242ac11000a.log HTTP/1.1" 500 95 "-" "python-requests/2.28.1"
The text was updated successfully, but these errors were encountered: