SSL certificate verify failed when running spacy download from CLI

[DTchebotarev]

Running

python -m spacy download en_core_web_lg

this fails with an SSL error. Other models, e.g. en_core_web_sm fail also. I'm behind a corporate proxy, though I've used the CLI download script without issue in the past.

Edit: I should add that downloading from github directly using pip seems to work following the directions here.

Traceback (most recent call last):
  File "/home/int/tche/.conda/envs/d/lib/python3.6/urllib/request.py", line 1318, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/home/int/tche/.conda/envs/d/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/int/tche/.conda/envs/d/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/int/tche/.conda/envs/d/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/int/tche/.conda/envs/d/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/home/int/tche/.conda/envs/d/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/home/int/tche/.conda/envs/d/lib/python3.6/http/client.py", line 1400, in connect
    server_hostname=server_hostname)
  File "/home/int/tche/.conda/envs/d/lib/python3.6/ssl.py", line 401, in wrap_socket
    _context=self, _session=session)
  File "/home/int/tche/.conda/envs/d/lib/python3.6/ssl.py", line 808, in __init__
    self.do_handshake()
  File "/home/int/tche/.conda/envs/d/lib/python3.6/ssl.py", line 1061, in do_handshake
    self._sslobj.do_handshake()
  File "/home/int/tche/.conda/envs/d/lib/python3.6/ssl.py", line 683, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:748)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/int/tche/.conda/envs/d/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/int/tche/.conda/envs/d/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/int/tche/.conda/envs/d/lib/python3.6/site-packages/spacy/__main__.py", line 31, in <module>
    plac.call(commands[command], sys.argv[1:])
  File "/home/int/tche/.conda/envs/d/lib/python3.6/site-packages/plac_core.py", line 328, in call
    cmd, result = parser.consume(arglist)
  File "/home/int/tche/.conda/envs/d/lib/python3.6/site-packages/plac_core.py", line 207, in consume
    return cmd, self.func(*(args + varargs + extraopts), **kwargs)
  File "/home/int/tche/.conda/envs/d/lib/python3.6/site-packages/spacy/cli/download.py", line 30, in download
    shortcuts = get_json(about.__shortcuts__, "available shortcuts")
  File "/home/int/tche/.conda/envs/d/lib/python3.6/site-packages/spacy/cli/download.py", line 55, in get_json
    data = url_read(url)
  File "/home/int/tche/.conda/envs/d/lib/python3.6/site-packages/spacy/compat.py", line 82, in url_read
    file_ = url_open(url)
  File "/home/int/tche/.conda/envs/d/lib/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/home/int/tche/.conda/envs/d/lib/python3.6/urllib/request.py", line 526, in open
    response = self._open(req, data)
  File "/home/int/tche/.conda/envs/d/lib/python3.6/urllib/request.py", line 544, in _open
    '_open', req)
  File "/home/int/tche/.conda/envs/d/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/home/int/tche/.conda/envs/d/lib/python3.6/urllib/request.py", line 1361, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/home/int/tche/.conda/envs/d/lib/python3.6/urllib/request.py", line 1320, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:748)>

Info about spaCy

• spaCy version: 2.0.11
• Platform: Linux-3.10.0-327.36.3.el7.x86_64-x86_64-with-redhat-7.2-Maipo
• Python version: 3.6.2

[ines]

Apologies for not having shipped the fix to this yet – you can find the solution for the current setup in #2212.

In the last version of spaCy, we replaced requests with urllib to cut down our dependencies a bit. But it turns out that this solution causes problems like this, because it doesn't look for SSL certificates in other places anymore (at least, that was our analysis, if I remember correctly). So we've decided to revert that change for the next version, to make things simpler again.

[DTchebotarev]

Ok, thanks. It seems this is fixed in the next version and there's a workaround for 2.0.11 so I will close the issue.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.