python 3 support

[BurntSushi]

Python 3 support is intricately linked with Python 3 support in nflgame. Whatever decision is made for nflgame will carry to nfldb.

See BurntSushi/nflgame#78

[BurntSushi]

This didn't happen. I'd still like to do it, but since we are close to the start of the season, it will have to remain Python 2 compatible to avoid major disruptions during the season. Thankfully, nfldb was written to be a little more conscious about Python 3. nflgame is where the fun lay.

[albertlyu]

I'm curious if @BurntSushi or @ochawkeye were able to install the dependencies for nfldb on Python 2 on a machine that has support for both Python 2 and Python 3? On my MacBook, Python 2.7 is pre-installed, so I can install nfldb just fine and run some example Python scripts. On my Windows computer though, I'm having a lot of trouble installing psycopg2 while having both Python 2.7 and 3.3 supported. I followed @ochawkeye's great instructions on installing on Windows, but manual installation detects that I have Python 3.3, while I need the Python 2.7 version of the PostgreSQL driver.

At the same time, I'm too stubborn to uninstall 3.3 and have 2.7 be my default Python version, since I have 3.3-based projects. I feel like the pip package manager will be all messed up too if I go that route. Any thoughts on how I can resolve this? Just wish that Python 2 and 3 can just get along :(

[BurntSushi]

@albertlyu It has been years since I haven't been on a system with only one version of Python. (My current system has 2.6, 2.7, 3.3 and 3.4 I think. Blech.)

Anyway, the versions are all pretty distinct. If you're running into version problems, it's usually because you're using Python X tools to install Python Y libraries. For example, you're probably trying to use pip---which is by default using Python 3---to install psycopg2, so it's not installing it for Python 2. You'll need to explicitly use pip for Python 2. On Linux systems, this usually means invoking pip2 or pip2.7. On Windows? I'm not sure what it will be called, but you've got to find a way to use the Python 2.7 version of pip.

Now that I say this though, it occurs to me that you probably can't install psycopg2 through pip on Windows. Can't you just download the Python 2.7 version of psycopg2 and install that? What happens when you try? (Please provide as much detail as possible.)

[albertlyu]

I've already tried pip2.7 install psycopg2 and that fails on error: Unable to find vcvarsall.bat, which I believe is the same error that other Windows users get. What I did was follow @ochawkeye's instructions to download psycopg2‑2.5.3.win‑amd64‑py2.7.exe (64-bit, Python 2.7), and when I click next in the installer, it tells me that Python 2.7 is required (since it's likely detecting that my default version is Python 3.3):

What I probably need to do is trick the installer into thinking that Python 2.7 is my primary Python version without uninstalling Python 3.3. I'm sure there's a way to manage the default python version? Such that when I type python -V in command line, it returns Python 2.7.6 instead of Python 3.3.2? Currently I have to type py -2 -V to invoke Python 2.7. Any additional thoughts?

[albertlyu]

Also, I set up my environment variables and Path to search C:\Python27;C:\Python27\Scripts instead of the Python 3.3 directories. I thought that might help? It may or may not.

[BurntSushi]

Changing your environment PATH sounds promising, but after a quick Google search, I'm skeptical. I am not a Windows guy at all and I don't know the first thing about a proper Windows Python environment.

The link seems to imply that the "default" Python version is stored in the registry somewhere. Blech. Which means changing the PATH probably won't do anything for binary installs.

More Googling. I found this:

32 bit:

pip install git+https://github.com/nwcell/psycopg2-windows.git@win32-py27#egg=psycopg2

64 bit:

pip install git+https://github.com/nwcell/psycopg2-windows.git@win64-py27#egg=psycopg2

[albertlyu]

It looks like the links to the GitHub project branches are out of date. https://github.com/nwcell/psycopg2-windows.git@win32-py27#egg=psycopg2 I'm going to try to figure out how to pip install the 64-bit, Python 2.7 version of that project.

[BurntSushi]

@albertlyu Did you actually try those commands? Note that it is using git+https:// and not https://. I don't think you can rely on the fact that those URLs don't exist in your browser.

[albertlyu]

Yup I did, sorry, I should have listed what I did. I tried the commands for the 64-bit version, and here's the output:

However, this is all I see in the site-packages directory of that package:

And trying to run top-ten-qbs.py in Python 2 produces the following:

So my assumption was that if I couldn't access the URL via https://, then it was broken. That was a wrong assumption, but it seems like I wasn't able to successfully install psycopg2 via that project. Unless I did something wrong? I'll keep looking...

[albertlyu]

BTW if I go up a level to the C:\Python27\Lib\site-packages directory, I don't see any folder for psycopg2. Also checked the Python33 folder, and don't see it there either.

I read a bit more of the psycopg2-windows project, and it appears that psycopg2 is only available in a virtual environment? I see that running that command cloned the package into a temp folder.

[ochawkeye]

I read a bit more of the psycopg2-windows project, and it appears that psycopg2 is only available in a virtual environment?

Hmm - not so sure what that's all about...

[BurntSushi]

If something works in a virtualenv then it has to work with regular pip too.

In any case, this page looks promising too. Note that you do not run the executable like normal! Instead, it wants you to use easy_install. Just make sure you use the easy_install for Python 2.7... (You shouldn't need to run the activate.bat step. That's for initializing a virtualenv.)

[ochawkeye]

Naive question - is there anything that gets registered in the Windows world when a package gets installed from pip, or is this as simple as me zipping up my psycopg2 folder and hosting it somewhere for @albertlyu to download and extract into his site-packages folder?

Obviously future updates would probably not be available through pip, but the snapshot should work for today's point in time, no?

[BurntSushi]

@ochawkeye Interesting question. Definitely not naive. In the Linux world, I imagine that would work as long as we were using machines on the same architecture. It's worth a shot.

[ochawkeye]

same architecture

There's the rub. I'm using 32, he's using 64...

We're in luck - I've got a virgin Lenovo laptop for test purposes. I've got to get it through Windows OOBE, but should be able to get @albertlyu what we're looking for to try.

[BurntSushi]

You could also get pip install psycopg2 working if you've configured a C++ compiler like mingw. http:https://stackoverflow.com/questions/2817869/error-unable-to-find-vcvarsall-bat --- It looks pretty hellish though.

[albertlyu]

I tried to easy_install the 64-bit version of psycopg2 from Stickpeople.com -- am I crazy or did the file labeled 64-bit actually install the 32-bit version? Anyway, here is what my command line looks like when I install the '64-bit' version:

And when I install the actual 32-bit version... it worked!!!

I definitely have a 64-bit operating system though, so not sure why that worked... but I'm not going to ask any more questions :) anyway thanks for your help!!

[BurntSushi]

@albertlyu Holy moly, yay!

I definitely have a 64-bit operating system though

Uh, it's not whether your OS is 64-bit or not, it's whether your Python installation is 64 bit. In the Windows world, I'm quite certain you can freely install 32 bit applications and be perfectly OK.

[ochawkeye]

Sure sure. Now makes sense why 64b installer couldn't find location...it didn't actually exist. 64b OS doesn't require 64b Python. I'm running 64 OS with 32b Python myself.

[albertlyu]

Oh wow, I never knew that. Running the Python interpreter just now shows that I have 32-bit Python 2.7 and 32-bit Python 3.3. I always assumed that since my OS is 64-bit, my installer was also 64-bit. Learn something new every day! Thanks guys, that was really helpful.

[BurntSushi]

@albertlyu No problem! Don't be a stranger. :-)