Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ssl-issues] OpenSSL/SSL Issues for Python 2.7.6 users #377

Closed
Make42 opened this issue Jul 11, 2015 · 28 comments
Closed

[ssl-issues] OpenSSL/SSL Issues for Python 2.7.6 users #377

Make42 opened this issue Jul 11, 2015 · 28 comments

Comments

@Make42
Copy link
Contributor

Make42 commented Jul 11, 2015

Hello, I still get the same errors as a couple of months ago:

  $ coursera-dl -u -p regmods-030
  Downloading class: regmods-030
  Starting new HTTPS connection (1): class.coursera.org
  /home/me/.local/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
  Starting new HTTPS connection (1): www.coursera.org
  /home/me/.local/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
  Traceback (most recent call last):
  File "/home/me/.local/bin/coursera-dl", line 9, in
  load_entry_point('coursera==0.0.3', 'console_scripts', 'coursera-dl')()
  File "/home/me/.local/lib/python2.7/site-packages/coursera/coursera_dl.py", line 944, in main
  result = download_class(args, class_name)
  File "/home/me/.local/lib/python2.7/site-packages/coursera/coursera_dl.py", line 842, in download_class
  username=args.username, password=args.password)
  File "/home/me/.local/lib/python2.7/site-packages/coursera/cookies.py", line 337, in get_cookies_for_class
  get_authentication_cookies(session, class_name, username, password)
  File "/home/me/.local/lib/python2.7/site-packages/coursera/cookies.py", line 160, in get_authentication_cookies
  login(session, username, password, class_name=class_name)
  File "/home/me/.local/lib/python2.7/site-packages/coursera/cookies.py", line 115, in login
  headers=headers, allow_redirects=False)
  File "/home/me/.local/lib/python2.7/site-packages/requests/sessions.py", line 508, in post
  return self.request('POST', url, data=data, json=json, *kwargs)
  File "/home/me/.local/lib/python2.7/site-packages/requests/sessions.py", line 465, in request
  resp = self.send(prep, *send_kwargs)
  File "/home/me/.local/lib/python2.7/site-packages/requests/sessions.py", line 573, in send
  r = adapter.send(request, **kwargs)
  File "/home/me/.local/lib/python2.7/site-packages/requests/adapters.py", line 431, in send
  raise SSLError(e, request=request)
  requests.exceptions.SSLError: [Errno 1] _ssl.c:510: error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure

I still have Python 2.7.6 on Ubuntu 14.4. I know it would be better to have Python 2.7.10 but this will not change anytime soon, since this is just the Python version that is in Ubuntu 14.4 has. PLEASE, can this be fixed, so that it works with python 2.7.6 ?

(As a matter of fact I even tried installing Python 2.7.10, and even though

  $ python --version
  Python 2.7.10

I get the same error.)

@Make42
Copy link
Contributor Author

Make42 commented Jul 18, 2015

Any development on this part?

@rbrito
Copy link
Member

rbrito commented Jul 20, 2015

Hi, @Make42 and anybody else having problems with SSL connections, Python
2.7 and so on.

On Jul 18 2015, Make42 wrote:

Any development on this part?

Are you willing to help me help you? :) I need you to perform some tests, so
that I can understand better the issue that we have at hand.

I wrote an (unfinished) document with some instructions to see where the
problem lies with people running older versions of Python/Ubuntu. I am
putting the document below my comments. Can you please perform the tests
that I describe in the document and and let me know:

1 - Does your system fail any of the tests?
2 - Are the instructions clear enough to be followed?

Please, again, keep in mind that the text is incomplete and that I would
like any help that I can get. I am the only developer of this project and I
need all the help that I can get from the users, so that I can keep the
project up and running for both the users and for myself.

BTW, I use myself the program and I don't see the bug (or I would have fixed
it already).

Anyway, here goes the text with the tests:


Test 1

You can perform some basic tests to see if your system openssl library is
good to work with coursera-dl.

If your system came with Python pre-installed (e.g., Linux, MacOS X or some
of the *BSDs), then it is very likely that your Python interpreter uses your
system openssl for encrypted connections (which is mandatory to know if
you can connect to Coursera or not via any Python program).

To test this, open a terminal window and type the following commands:

openssl s_client -ssl3 -connect class.coursera.org:443

The first one should give you immediately an error that starts with some
lines like:

CONNECTED(00000003)
140107945010848:error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure:s3_pkt.c:1262:SSL alert number 40
140107945010848:error:1409E0E5:SSL routines:SSL3_WRITE_BYTES:ssl handshake failure:s3_pkt.c:598:

The specific numbers at the begininng of the line are not important, but the
parts of handshake failure are.

Now, type:

openssl s_client -tls1 -connect class.coursera.org:443

If it works well, it will print many lines, ending with something like:

Start Time: 1432413833
Timeout   : 7200 (sec)
Verify return code: 20 (unable to get local issuer certificate)
---

and waiting there for you to type more commands. You can terminate the
execution of openssl with a simple Control-C.

If that's what you get, please read the next step. If you get something
different from what is described above, please let us know via our issue
tracker: https://github.com/coursera-dl/coursera/issues

Test 2

In a Python (just type python or python3 in a command line/terminal)
interpreter, type the following commands, one per line:

import ssl
from ssl import wrap_socket, CERT_NONE, PROTOCOL_SSLv23
from ssl import HAS_SNI

If your installation is good, then you should see your Python interpreter
simply be waiting for more commands with no extra output other than what you
type. If your installation is not good, then you will likely get an error
message from your Python interpreter with the last line starting with
ImportError.

If this is the case, you are strongly advised to install a newer version of
the Python interpreter. If you can, then it will be good to install the most
recent stable version of Python 3 that is available at
https://www.python.org/downloads/

Test 3

In a Python (just type python or python3 in a command line/terminal)
interpreter, type the following commands, one per line:

import requests
r = requests.get('https://class.coursera.org')
r.status_code, r.text[:15]

If your installation is good, then you should see your Python interpreter
respond with something like after you type the 3rd line above:

(200, u'<!DOCTYPE html>')

If you get an error message like

Test 4


As you can see, the text is unfinished, but I would love to know the answers
to them so far. The more information that I get, the better that I can help
you.

Thanks,

Rogério Brito : rbrito@{ime.usp.br,gmail.com} : GPG key 4096R/BCFCAAAA
http:https://cynic.cc/blog/ : github.com/rbrito : profiles.google.com/rbrito
DebianQA: http:https://qa.debian.org/developer.php?login=rbrito%40ime.usp.br

@Make42
Copy link
Contributor Author

Make42 commented Jul 20, 2015

$  openssl s_client -ssl3 -connect class.coursera.org:443
CONNECTED(00000003)
140445491275424:error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure:s3_pkt.c:1262:SSL alert number 40
140445491275424:error:1409E0E5:SSL routines:SSL3_WRITE_BYTES:ssl handshake failure:s3_pkt.c:598:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 0 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : SSLv3
    Cipher    : 0000
    Session-ID: 
    Session-ID-ctx: 
    Master-Key: 
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1437422284
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
---

Because I tried to install Python 2.7.10, my entire python is completely f**ed. Thus I am not able to run the other commands. I wrote a question on the issue on http:https://askubuntu.com/questions/648424/moun-is-gone-after-change-of-python-issues-after-python-2-7-10-installation-on but it seems nobody really cares :.-( (or knows anything).

Also: #288 and #370

@rbrito
Copy link
Member

rbrito commented Jul 20, 2015

Hi.

On Jul 20 2015, Make42 wrote:

$ openssl s_client -ssl3 -connect class.coursera.org:443
(...)

Excellent. What about the result of the other openssl command? (The one
where you specify -tls1 instead of -ssl3).

Because I tried to install Python 2.7.10, my entire python is completely
f**ed.

OK, I see. Let's try to fix things one by one.

Thus I am not able to run the other commands. I wrote a question on the
issue on
http:https://askubuntu.com/questions/648424/moun-is-gone-after-change-of-python-issues-after-python-2-7-10-installation-on
but it seems nobody really cares :.-( (or knows anything).

I will try to elaborate it there (I already upvoted your question there, so
that you got a few more points so that you can do things like comments or
something similar).

I hope that I will get you to "unbreak" your installation, but you will have
to provide me with a lot of details (I have some experience here, as I have
"unbroken" many installations before due to failing hardware/data loss---and
I have also migrated many systems from Ubuntu to Debian too, without
reinstallation). :)

Of course, it is fun to do that, but only if you want to learn the process.
If you are really in a hurry, be practical, create a chroot/virtual
machine/container and get the work done.

Otherwise, relax and try to get your system fixed (but get your work done
before).

Do you have any instant messaging program? You can send me it via private
e-mail, if you want. I will need to ask you many, many questions so that I
can understand what is happening.

Hope this helps,

Rogério Brito : rbrito@{ime.usp.br,gmail.com} : GPG key 4096R/BCFCAAAA
http:https://cynic.cc/blog/ : github.com/rbrito : profiles.google.com/rbrito
DebianQA: http:https://qa.debian.org/developer.php?login=rbrito%40ime.usp.br

@Make42
Copy link
Contributor Author

Make42 commented Jul 21, 2015

Ok, the reporting is fine by me. Thank you very much! I am happy to learn this, since it seems to me, this is something worth knowing (how dpkg and and linux internals work and how to fix them if they don't).

Since it is already the second time this year that I broke my system because of coursera-dl (last time I reinstalled) I am not so keen of reinstalling again. ;-)

Since the notebook in question is my private machine and my business notebook is unaffected, I can "get my work done" without interference. Also, I downloaded the coursera courses the docker-way - so that worked.

As soon as I am at my private machine I will try out the other ssl command.

I have skype and jitsi.

I am waiting for your list of commands I have to use to provide you with information at askubuntu, right?

Btw.: I wrote an email to [email protected]

@jordan-rosenfeld
Copy link

I'm having this same issue (also Ubuntu 14.04). Once Python 3.4 is installed, but not set as default, how would you run coursera-dl? The default is still Python 2.7.6, but I don't want to change that in order to avoid breaking the entire system (as highlighted above).

@Make42
Copy link
Contributor Author

Make42 commented Jul 25, 2015

@rbrito I haven't gotten any answer/questions from you...

@NelsonMinar
Copy link

You may find the discussion in issue #330 useful. The underlying problem is Ubuntu 14.04 includes Python 2.7.6, which has some SSL compatibility problems. The simple Ubuntu-friendly option is to install python3 and pip3 via Ubuntu's apt package manager, then do pip3 install coursera. That will make coursera-dl run under Python 3 which will work in Ubuntu 14.04.

@Make42
Copy link
Contributor Author

Make42 commented Jul 28, 2015

@NelsonMinar Thanks for your comment. However, this will not do anymore. Not regarding the state of my System. Its python is completely damaged. See http:https://unix.stackexchange.com/questions/218911/help-me-repair-my-python-i-think-dpkg-has-to-be-reset-very-tricky-issue Edit: It got solved!

I might try your solution later.

@sagunms
Copy link

sagunms commented Jul 30, 2015

@Make42 Python 2.7 pip didn't work for me in both Mac and Ubuntu systems. The following steps worked for me:

Mac OSX (Yosemite):

brew install python3
pip3 install coursera

Ubuntu 14.04 LTS

sudo apt-get install python3 python3-pip
sudo pip3 install coursera

Usage Example

cd ~/Downloads/MyAwesomeCourses
coursera-dl -u [email protected] -p 'yourPA$SwORd' algorithmicthink2-002

Hope it helps.

@rbrito
Copy link
Member

rbrito commented Aug 3, 2015

Hi, @Make42.

On Aug 02 2015, Make42 wrote:

@sagunms: I tried, but got:

  $ coursera-dl --clear-cache -u XXXX -p XXXX datavisualization-001
  Downloading class: datavisualization-001
  Starting new HTTPS connection (1): class.coursera.org
  Starting new HTTPS connection (1): www.coursera.org
  Logged in on coursera.org.
  Starting new HTTPS connection (1): class.coursera.org
  Found authentication cookies.
  Downloaded https://class.coursera.org/datavisualization-001/lecture/index (14546 bytes)
  Found 0 sections and 0 lectures on this page
  The cookies file may be invalid, please re-run with the `--clear-cache` option.

I'm sending you a private e-mail so that we agree on a date to fix your
system installation.

After that, we can try to see this other issue, which seems unrelated (which
version of python are you using now, BTW?).

Regards,

Rogério Brito : rbrito@{ime.usp.br,gmail.com} : GPG key 4096R/BCFCAAAA
http:https://cynic.cc/blog/ : github.com/rbrito : profiles.google.com/rbrito
DebianQA: http:https://qa.debian.org/developer.php?login=rbrito%40ime.usp.br

@Make42
Copy link
Contributor Author

Make42 commented Aug 4, 2015

@rbrito You are too late to save my machine ;-). I was able to get the issue solved (see the edit above).

Also (I just tried it out)

  sudo apt-get install python3 python3-pip
  sudo pip3 install coursera

works for me indeed. Maybe this could be put into the README.md... ?! I forked the repo added the changed and made a pull request (it's my first time on github - I thought it might be something small to try this out in the wild.)

@rbrito
Copy link
Member

rbrito commented Aug 14, 2015

Hi.

I'm putting the text that I sent before via e-mail, since github thought that it was just a quote and made an ellipsis out of it.

These are some diagnostics that I would appreciate if anybody could run (especially those that experienced the issue, solved/worked-around-it, but still can reproduce it), so that we can better understand the issue.

(Oh, improvements to this text are highly welcome; Yes, contributing to our project (especially with documentation) is easy and it's never too late to get started!).

Anyway, here goes the text with the tests:


Test 1

You can perform some basic tests to see if your system openssl library is
good to work with coursera-dl.

If your system came with Python pre-installed (e.g., Linux, MacOS X or some
of the *BSDs), then it is very likely that your Python interpreter uses your
system openssl for encrypted connections (which is mandatory to know if
you can connect to Coursera or not via any Python program).

To test this, open a terminal window and type the following commands:

openssl s_client -ssl3 -connect class.coursera.org:443

The first one should give you immediately an error that starts with some
lines like:

CONNECTED(00000003)
140107945010848:error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure:s3_pkt.c:1262:SSL alert number 40
140107945010848:error:1409E0E5:SSL routines:SSL3_WRITE_BYTES:ssl handshake failure:s3_pkt.c:598:

The specific numbers at the begininng of the line are not important, but the
parts of handshake failure are.

Now, type:

openssl s_client -tls1 -connect class.coursera.org:443

If it works well, it will print many lines, ending with something like:

Start Time: 1432413833
Timeout   : 7200 (sec)
Verify return code: 20 (unable to get local issuer certificate)
---

and waiting there for you to type more commands. You can terminate the
execution of openssl with a simple Control-C.

If that's what you get, please read the next step. If you get something
different from what is described above, please let us know via our issue
tracker: https://github.com/coursera-dl/coursera/issues

Test 2

In a Python (just type python or python3 in a command line/terminal)
interpreter, type the following commands, one per line:

import ssl
from ssl import wrap_socket, CERT_NONE, PROTOCOL_SSLv23
from ssl import HAS_SNI

If your installation is good, then you should see your Python interpreter
simply be waiting for more commands with no extra output other than what you
type. If your installation is not good, then you will likely get an error
message from your Python interpreter with the last line starting with
ImportError.

If this is the case, you are strongly advised to install a newer version of
the Python interpreter. If you can, then it will be good to install the most
recent stable version of Python 3 that is available at
https://www.python.org/downloads/

Test 3

In a Python (just type python or python3 in a command line/terminal)
interpreter, type the following commands, one per line:

import requests
r = requests.get('https://class.coursera.org')
r.status_code, r.text[:15]

If your installation is good, then you should see your Python interpreter
respond with something like after you type the 3rd line above:

(200, u'<!DOCTYPE html>')

If you get an error message like

Test 4


As you can see, the text is unfinished, but I would love to know the answers
to them so far. The more information that I get, the better that I can help
you.

@haisum
Copy link

haisum commented Aug 15, 2015

This is re-post of my comment from issue #383 with error log and some more info.

I am using Ubuntu 14.04 LTS 64 bit. Default python version is 2.7.6. When I run pip install coursera and run coursera-dl I get following error:

vagrant@python:~$ coursera-dl -u username@gmail.com -p password --on-demand machine-learning
Downloading class: machine-learning
Starting new HTTPS connection (1): www.coursera.org
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
Traceback (most recent call last):
  File "/usr/local/bin/coursera-dl", line 9, in <module>
    load_entry_point('coursera==0.0.3', 'console_scripts', 'coursera-dl')()
  File "/usr/local/lib/python2.7/dist-packages/coursera/coursera_dl.py", line 942, in main
    result = download_on_demand_class(args, class_name)
  File "/usr/local/lib/python2.7/dist-packages/coursera/coursera_dl.py", line 885, in download_on_demand_class
    login(session, args.username, args.password)
  File "/usr/local/lib/python2.7/dist-packages/coursera/cookies.py", line 115, in login
    headers=headers, allow_redirects=False)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 508, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 465, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 573, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 431, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: [Errno 1] _ssl.c:510: error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure

A temporary workaround I figured out was using python3 via virtualenv because it isn't easy to upgrade python's default version in ubuntu, it fixes one thing and breaks many. Here's workaround that worked for me:

sudo easy_install virtualenv
virtualenv -p /usr/bin/python3 coursera
cd coursera/
./bin/pip install coursera
./bin/coursera-dl -u [email protected] -p password --on-demand machine-learning

@rbrito if you want any more info or logs, I would be glad to help.

@rbrito
Copy link
Member

rbrito commented Aug 16, 2015

@haisum, first of all, thank you so very much for both your willingness to help and your detailed response.

I have other questions (and these are valid for every other poster/subscriber of this bug report---the more answers that we get, the more likely that I am to see the pattern---hopefully):

1 - What is the version of Python 3 that you are using? You can find it via the following command:

dpkg -s python3 | grep Version:

2 - Do the problems that you describe persist if you use a virtual environment with your default Python 2 interpreter?

3 - Can you run the tests that I wrote about in #377 (comment) with your Python 2 interpreter outside of any virtual environment? I would really love to know what may be happening and where the issues may lie.

4 - Are you willing to compile a short module for your Python 2 interpreter inside a virtualenv? If yes (and that would help a lot), then I would love it if you, first, installed (regularly, outside of a virtualenv) the following packages in your Ubuntu:

sudo apt-get install build-essential python-dev libssl-dev libffi-dev

Then, create new a virtual environment with Python 2 (as usual), enter it, install everything as you would (e.g., inside the virtual environment type pip install coursera) and, after everything is in place, install (still inside the virtualenv) the following python modules:

pip install urllib3 pyasn1 ndg-httpsclient pyOpenSSL

Some of the packages above may be already installed (they were likely installed when you run pip install coursera), so any message that they are there already are harmless.

I have some more questions that I would like to have answered, but let's take those one at a time. :)

Thanks a lot,

Rogério.

@mxamin
Copy link
Contributor

mxamin commented Aug 16, 2015

@rbrito, I installed Ubuntu 14.04 on a virtual machine and followed your instruction about installing new modules:

sudo apt-get install build-essential python-dev libssl-dev libffi-dev
pip install urllib3 pyasn1 ndg-httpsclient pyOpenSSL

Fortunately, It worked and the error never appeared again. I also test it on windows with Python 2.7.6 already installed and got the same result by only installing ndg-httpsclient module, nothing more.
As a matter of fact installing all those other packages in Ubuntu is because of cryptography module!

@haisum
Copy link

haisum commented Aug 16, 2015

@rbrito

Here are the results:

What is the version of Python 3 that you are using?

Version: 3.4.0-0ubuntu2

Do the problems that you describe persist if you use a virtual environment with your default Python 2 interpreter?

Yes.

Can you run tests?

First Test

Everything fine.

Second Test
vagrant@python:~/coursera2$ python
Python 2.7.6 (default, Mar 22 2014, 22:59:56) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>> from ssl import wrap_socket, CERT_NONE, PROTOCOL_SSLv23
>>> from ssl import HAS_SNI
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name HAS_SNI

Also note that advice of upgrading default python, as given in test 2 text, on Ubuntu is not practical in most of cases because ubuntu heavily relies on python and upgrade may cause failures elsewhere on system.

Third Test
>>> r = requests.get("https://class.coursera.org")
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 69, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 50, in request
    response = session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 465, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 594, in send
    history = [resp for resp in gen] if allow_redirects else []
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 196, in resolve_redirects
    **adapter_kwargs
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 573, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 431, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: [Errno 1] _ssl.c:510: error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure

Willing to compile short module?

sudo apt-get install build-essential python-dev libssl-dev libffi-dev 

virtualenv -p /usr/bin/python coursera

cd coursera

./bin/pip install urllib3 pyasn1 ndg-httpsclient pyOpenSSL
/home/vagrant/coursera3/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
Collecting urllib3
/home/vagrant/coursera3/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
  Using cached urllib3-1.11-py2.py3-none-any.whl

@haisum
Copy link

haisum commented Aug 16, 2015

I was able to solve issue with stock python by upgrading ndg-httpsclient module by using following command:

sudo pip install --upgrade ndg-httpsclient 

This fixes the issue on default python without upgrading it.

Source: Jessica's answer on http:https://stackoverflow.com/questions/29099404/ssl-insecureplatform-error-when-using-requests-package

@dentarthur
Copy link

FYI

  1. I am having similar problem with most recent version of same course regmod-032 (they do monthly course sessions) and also with repdata-032 but not with earlier courses in the Data Science Specialization at Coursera. Conceivably course staff rather than coursera doing something new?

  2. Python 2.7.5 (not 2.7.6)

  3. Slight variation, no "insecure platform warning" and problem started AFTER successfully listing the index, just before first actual download:

Downloading https://class.coursera.org/regmods-032/lecture/download.mp4?lecture_id=171 -> /run/media/admin/My Book/CURRENTcoursera/regmods-032/01_Week_1/01_01_01_Part_1_of_4_Introduction_to_Regression_6-03.mp4
Starting new HTTPS connection (1): d28rh4a8wq0iu5.cloudfront.net
Traceback (most recent call last):
File "./get", line 6, in
coursera_dl.main()
File "/home/admin/Documents/coursera/coursera_dl.py", line 781, in main
if download_class(args, class_name):
File "/home/admin/Documents/coursera/coursera_dl.py", line 761, in download_class
args.intact_fnames)
File "/home/admin/Documents/coursera/coursera_dl.py", line 400, in download_lectures
downloader.download(url, lecfn)
File "/home/admin/Documents/coursera/downloaders.py", line 43, in download
self._start_download(url, filename)
File "/home/admin/Documents/coursera/downloaders.py", line 280, in _start_download
r = self.session.get(url, stream=True)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 473, in get
return self.request('GET', url, *_kwargs)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 461, in request
resp = self.send(prep, *_send_kwargs)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 599, in send
history = [resp for resp in gen] if allow_redirects else []
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 192, in resolve_redirects
allow_redirects=False,
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 573, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 431, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: [Errno 1] _ssl.c:504: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

  1. Above fix did not fix it:
    sudo pip install --upgrade ndg-httpsclient

  2. I am using no longer supported Fedora 20 and won't have time to upgrade for a while.
    After skim reading the whole thread, which I cannot do today, I may try compiling latest python source directly and/or installing latest coursera-dl.

  3. Let me know if you want me to use any particular compile flags or any specific coursera-dl and run any specific tests in the thread. I won't be reading the thread carefully myself yet but will probably wait a couple of days before doing above in case there is any such specific request (or please add a note that there isn't so I stop waiting).

  4. Be VERY specific about precise compiler flags and logging required for python etc. I don't know much.

@balta2ar
Copy link
Member

@rbrito What's the status on this issue? Could you please summarize? Should we catch the error in the code and print a message that recommends to install ndg-httpsclient?

@dentarthur
Copy link

Scrub my suggestion above that problem could conceivably be specific to ..."the Data Science Specialization at Coursera. Conceivably course staff rather than coursera doing something new?"

I now assume it was just coincidence that my report was from same series of monthly repeated courses as the original report at top of thread.

I previously thought it was not occurring for me with several other coursera courses I was still downloading. But they may have been completed with no actual file downloads but just successfully checking the index pages.

Just encounted the same problem with unrelated coursera course and it only happens AFTER index page successfully obtained, at first actual file download:

Downloading class: valuation-001
Starting new HTTPS connection (1): class.coursera.org
Starting new HTTPS connection (1): class.coursera.org
Starting new HTTPS connection (1): accounts.coursera.org
Logged in on accounts.coursera.org.
Found authentication cookies.
Downloaded https://class.coursera.org/valuation-001/lecture/index (33456 bytes)
Module_one_videos
1.0_Welcome_and_Introduction_2-30
1.1_Measuring_stand-alone_risk_Just_the_same_old_standard_deviations_13-55
1.2_Alternative_attitudes_towards_risk_I_dont_hate_risk_Im_just_averse_to_it..._13-57
1.3_Portfolio_Return_and_Risk_The_more_the_merrier_11-15
1.4_Defining_the_Diversification_Benefit_Its_All_About_the_Eggs_and_Baskets..._14-08
Found 1 sections and 5 lectures on this page
Downloading: /run/media/admin/My Book/CURRENTcoursera/valuation-001/01_Module_one_videos/01_1.0_Welcome_and_Introduction_2-30.pdf
Downloading https://d396qusza40orc.cloudfront.net/valuation/PDFs/004_001_000_Introduction_GMc_001.pdf -> /run/media/admin/My Book/CURRENTcoursera/valuation-001/01_Module_one_videos/01_1.0_Welcome_and_Introduction_2-30.pdf
Starting new HTTPS connection (1): d396qusza40orc.cloudfront.net
Traceback (most recent call last):
File "./get", line 6, in
coursera_dl.main()
File "/home/admin/Documents/coursera/coursera_dl.py", line 781, in main
if download_class(args, class_name):
File "/home/admin/Documents/coursera/coursera_dl.py", line 761, in download_class
args.intact_fnames)
File "/home/admin/Documents/coursera/coursera_dl.py", line 400, in download_lectures
downloader.download(url, lecfn)
File "/home/admin/Documents/coursera/downloaders.py", line 43, in download
self._start_download(url, filename)
File "/home/admin/Documents/coursera/downloaders.py", line 280, in _start_download
r = self.session.get(url, stream=True)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 473, in get
return self.request('GET', url, *_kwargs)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 461, in request
resp = self.send(prep, *_send_kwargs)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 573, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 431, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: [Errno bad handshake] [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')]

PS Have not done and now cannot do any testing due to sudden family emergency.

@t0mst0ne
Copy link

Just happen to have the same condition as @dentarthur ...
I had installed and updated ndg-httpsclient
The problem was solved by installing through pip3 with python 3

@balta2ar balta2ar changed the title OpenSSL/SSL Issues for Python 2.7.6 users [ssl-issues] OpenSSL/SSL Issues for Python 2.7.6 users Jun 12, 2016
@mehul-m-prajapati
Copy link

Hi @rbrito,

I am still facing this issue on Ubuntu 16.04 (Python3.5)

./coursera-dl --clear-cache -u [email protected] -p xxx machine-learning

coursera_dl version 0.10.0
Downloading class: machine-learning (1 / 1)
SSLError HTTPSConnectionPool(host='www.coursera.org', port=443): Max retries exceeded with url: /api/login/v3 (Caused by SSLError(SSLError(1, '[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:645)'),))

#####################################################################
# ATTENTION! PLEASE READ THIS!
#
# The following error has just occurred:
# SSLError HTTPSConnectionPool(host='www.coursera.org', port=443): Max retries exceeded with url: /api/login/v3 (Caused by SSLError(SSLError(1, '[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:645)'),))
#
# Please read instructions on how to fix this error here:
# https://github.com/coursera-dl/coursera-dl#sslerror-errno-1-_sslc504-error14094410ssl-routinesssl3_read_bytessslv3-alert-handshake-failure
#####################################################################

I have tried following and facing the issue,

virtualenv -p python3 my-coursera
cd my-coursera
source bin/activate
git clone https://github.com/coursera-dl/coursera
cd coursera
pip3 install -r requirements.txt

@Moulick
Copy link

Moulick commented Jun 2, 2018

Hey @rbrito thanks for making this. I tried coursera-dl today and am getting this error.

I am running python 3.6.5 in a virtualenv with pip install urllib3 pyasn1 ndg-httpsclient. I even tried the docker method with both 14.04 base and 16.04 base , no dice.

I have read both #330 and #377, no luck. All the tests you mentioned above pass. Any help please? If there is any more information you want, I am ready to give. I have attached the output of the --debug
Please help

https://pastebin.com/gTzeKyhp

@balta2ar
Copy link
Member

balta2ar commented Jun 2, 2018

@Moulick Could you try this pull request? #660

@mehul-m-prajapati
Copy link

The #660 worked like a charm. Thanks @balta2ar

@Moulick
Copy link

Moulick commented Jun 3, 2018

@balta2ar yep, solved by #660. But now subtitles are not downloaded unless I use --aria2

@balta2ar
Copy link
Member

balta2ar commented Jun 3, 2018

FTR, should be fixed in 0.11.2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests