Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot connect if behind a HTTP proxy #205

Closed
esalagea opened this issue Nov 19, 2013 · 7 comments
Closed

Cannot connect if behind a HTTP proxy #205

esalagea opened this issue Nov 19, 2013 · 7 comments

Comments

@esalagea
Copy link

The script works just fine if not behind a HTTP proxy. thanks a lot for this downloader, is the best I found for coursera.

If I run it behind my company's http proxy, I get:

./coursera-dl --debug -u userName -p pasword class-id

Here is the output

root[main] Downloading class: class-id
requests.packages.urllib3.connectionpool[_new_conn] Starting new HTTP connection (1): proxy.mycompany.com
requests.packages.urllib3.connectionpool[_make_request] "GET https://class.coursera.org/course-id HTTP/1.1" 200 None
root[main] Could not authenticate: Did not recieve csrf_token cookie.

@jonasdt
Copy link
Member

jonasdt commented Nov 20, 2013

We have seen this problem before and we never managed to get it to work :(
Can you set the HTTP_PROXY (and/or HTTPS_PROXY) environment variables and try again?

$ export HTTP_PROXY="http:https://proxy.mycompany.com"
$ export HTTPS_PROXY="https://proxy.mycompany.com"
$ ./coursera-dl --debug -u userName -p pasword class-id

If this works, we should add a proxy option to coursera-dl.

@esalagea
Copy link
Author

I tried it, same result. sorry :(

@Empty0ne
Copy link

You shouldn't need to add the proxy option to the script, it handles the
enviroment varaibles just fine. Per application proxy rules are a pain to
maintain for users if they have to access multiple networks with different
proxy configurations. I am using coursera-dl through a corporate proxy
without any issues.

Are you able to get out of the corporate network using any other
commandline programs like curl or wget?

If you try to run 'wget http:https://google.com' do you run into the same issue?

Depending on your distro, your env variables may be different. Using all
caps for me doesn't work, I have to use lower case, it may be the same for
you. Here is how I have my proxies set up in my .bashrc:

export http_proxy=http:https://PROXY_URL:PORT
export https_proxy=$http_proxy

this is the output I get:

~$ coursera-dl --debug -n ~/.netrc crypto-007
root[authenticate_through_netrc] Trying netrc file /home/empty/.netrc
root[main] Downloading class: crypto-007
requests.packages.urllib3.connectionpool[_new_conn] Starting new HTTPS
connection (1): class.coursera.org
requests.packages.urllib3.connectionpool[_make_request] "GET /crypto-007
HTTP/1.1" 302 0
requests.packages.urllib3.connectionpool[_new_conn] Starting new HTTPS
connection (1): accounts.coursera.org
requests.packages.urllib3.connectionpool[_make_request] "POST /api/v1/login
HTTP/1.1" 200 0
root[login] Logged in on accounts.coursera.org.
requests.packages.urllib3.connectionpool[_new_conn] Starting new HTTPS
connection (1): class.coursera.org
requests.packages.urllib3.connectionpool[_make_request] "GET
/crypto-007/auth/auth_redirector?type=login&subtype=normal HTTP/1.1" 302 0
requests.packages.urllib3.connectionpool[_make_request] "GET
/crypto-007/class HTTP/1.1" 200 11830
root[get_authentication_cookies] Found authentication cookies.
root[get_syllabus_url] Using index mode with page:
https://class.coursera.org/crypto-007/lecture/index
requests.packages.urllib3.connectionpool[_make_request] "GET
/crypto-007/lecture/index HTTP/1.1" 200 None
root[get_syllabus] Downloaded
https://class.coursera.org/crypto-007/lecture/index (176485 bytes)

On Wed, Nov 20, 2013 at 5:14 AM, esalagea [email protected] wrote:

I tried it, same result. sorry :(


Reply to this email directly or view it on GitHubhttps://github.com//issues/205#issuecomment-28879616
.

@jonasdt
Copy link
Member

jonasdt commented Nov 25, 2013

@esalagea did you try EmptyOne's suggestion?

@esalagea
Copy link
Author

I'm sorry. Maybe there is something something tricky in my configuration.
wget works just fine (I tried it on a coursera https page).
Exporting lowercase http_proxy did not help. Same output.

@Jacksonville
Copy link

For info, if using Windows you would need to use set.
eg:
set HTTP_PROXY=http:https://host:port
set HTTPS_PROXY=http:https://host:port

@rbrito
Copy link
Member

rbrito commented Aug 1, 2014

Hi. I am closing this issue. If the problem persists, please open a new one with further information and some context, so that we have a better basis to explore the problem.

@rbrito rbrito closed this as completed Aug 1, 2014
balta2ar added a commit that referenced this issue Mar 19, 2017
danielrodcaball pushed a commit to danielrodcaball/coursera-dl that referenced this issue Jul 2, 2018
In order to download courses behind a proxy server, referenced in the issues coursera-dl#205, coursera-dl#594 and coursera-dl#613, it's necesary get the proxy enviroment variables before do session.send() in file network.py. Due to some diferents behaviors in request module these variables was no used to make the request. Therefore, in order to get enviroment settings was added the line:

settings = session.merge_environment_settings(request.url, {}, False, False, None)

Then the request was made with these settings using:

reply = session.send(prepared_request, **settings)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants