Setting up virtualenv on mac

[karthiknadig]

Question from survey feedback:

I looked at that page, and this one:

https://code.visualstudio.com/docs/python/environments#_where-the-extension-looks-for-environments

But it's not clear what I have to do to make sure that VSCode can find virtualenvs. They are in the typical (for mac) virtualenv location of:

~/.virtualenvs/
And I use pyenv so the dropdown seems to see the python versions from pyenv, but those aren't taking into account attaching the virtualenv. So could you tell me which of these environment variables I need to set for ~/.virtualenvs/* to be found on MacOS?
Alternatively, I'm happy to use a Dockerfile to eliminate the need for pyenv and virtualenv, but it's not clear how that would work with VSCode.

[d3r3kk]

I believe that for this issue, the page linked has the information required to get this working. I am having trouble what might be unclear, as in the page linked we see:

Virtual environments located in the folder identified by the python.venvPath setting (see General settings). The extension looks for virtual environments in the first-level subfolders of venvPath.

And from our python.venvFolders setting we have:

  // Folders in your home directory to look into for virtual environments.
  "python.venvFolders": [
    "envs",
    ".pyenv",
    ".direnv"
  ],

If the user wants to see virtual environments on Mac then, she could simply add .virtualenvs to the python.venvFolders and presumably, the virtual environments would be found.

[d3r3kk]

@DonJayamanne do we want to add ~/.virtualenv to the list of default venvFolders?

[brettcannon]

That location isn't specifically typical as the OP claims as virtual environments have no default unless you use some tool that provides some default.

[brettcannon]

Closing as this isn't an issue with the extension but the OP not realizing they are using tools that are making assumptions.

[davidthewatson]

I'm the OP of the question @karthiknadig posted above.

In addition, I tried adding my virtualenv location to the configuration, but select interpreter still didn't find the virtualenvs. Is there some specific set of steps that I need to take to add the virtualenv paths and make sure that the config is re-parsed by VSCode? Merely saving the config does not seem to be updating the items in the select interpreter list.

I'd like to get this working on Mac as this is about the only thing that keeps me coming back to atom where the simplicity of atom . at the command line inherits the active virtualenv from the previous workon, which, while primitive, is reliable and immediately understandable when it breaks.

As a long time user of visual studio, it feels like VSCode is skating too close to the legacy Microsoft/Java config style here and needs to lean a bit in the command line direction. For instance, why can't I just vscode . from the command line and inherit the active environment?

[brettcannon]

@davidthewatson what do you mean by "tried adding my virtualenv location"? Are you talking about python.pythonPath or python.venvPath? If you specify the path to the Python interpreter in your virtual environment with python.pythonPath then it will be used as the Python interpreter. With python.venvPath that can be directory of multiple virtual environments and then we let you select the virtual environment you want.

As for picking up an explicitly activated environment, we probably could, it's just no one has really asked for it. 😄 I've opened #2124 to discuss the idea of reading the VIRTUAL_ENV environment variable when python.pythonPath is not already set.

[alexwhittemore]

I'm similarly lost, on a couple things. Pardon my ignorance - I thought I had vscode and python all figured out, but clearly I know nothing :)

On windows, for approx. ever now, my go-to workflow has been to use virtualenvwrapper-powershell with all projects. Something like

1. mkvirtualenv my_project_env
2. workon my_project_env
3. code .

This opens vscode, which inherits the virtualenv and all the linting/etc/etc uses what's installed in that venv. If I hit "install linter" on first run (when it's not installed yet), that gets installed into that virtualenv. Cool.

I thought this is how I was doing things on Mac too, but I'll admit, I haven't been on mac in a while. I'm now setting up a new machine, and when I try to do the same thing,
(my_project_env) $ code .
opens code in my project dir, just as I expect, but without the virtualenv loaded

So for one, does this workflow (terminal>cd to project dir>workon myenv>code .) simply not work on Mac like it does on windows with powershell? Can I make it work that way?

And for two - OP seems to be having trouble with shift-cmd-p>select interpreter not finding his virtualenvs in ~/.virtualenvs. My .bash_profile has had in it, for the longest time, export WORKON_HOME=$HOME/.virtualenvs, probably from some tutorial I followed way back when. But when I do the same thing in code to go looking for an interpreter, it finds all my virtualenvs in .virtualenvs just fine. But my settings are as-yet default, i.e.

// Folders in your home directory to look into for virtual environments.
  "python.venvFolders": [
    "envs",
    ".pyenv",
    ".direnv"
  ],

So why does that part work fine for me but not OP?

[davidthewatson]

Are you talking about python.pythonPath or python.venvPath

I can set pythonPath just fine. venvPath on the other hand, seems to not work at all. I've yet to see it come up with anything but the global environment on my mac, which is Apple's python 2.7, which, of course, I don't use since I use pyenv and run only in various python 3 environments.

I've gone through many attempts at this where I blank out both the user and workspace settings and start from a clean slate setting venvPath to no avail.

workspace settings:

{
    "python.venvPath": "~/.virtualenvs"
}

OR

{
    "python.venvFolders": [".virtualenvs"]
}

code . does not work for me as there is no script or binary exposed by code. I'm pretty sure this would be a solved problem if I knew what binary or script to invoke from the mac cmd line as the active virtualenv would be inherited.

Also, fwiw:

$ echo $WORKON_HOME
/Users/davidwatson/.virtualenvs

One thing I'm confused about is what is the difference or purpose of having venvFolders and venvPath?

If there's an easy way to hookup a docker image to each python project config, I'm happy to just abandon virtualenvs since I run all of this code on kubernetes dev, staging, prod environments anyhow and derive from python 3 images as a result.

[alexwhittemore]

code . does not work for me as there is no script or binary exposed by code. I'm pretty sure this would be a solved problem if I knew what binary or script to invoke from the mac cmd line as the active virtualenv would be inherited.

That I can help with: fire up code and shift-cmd-P "Install code command in path" - that'll add a symlink at /usr/local/bin/code

But it doesn't seem to be the case that it inherits virtualenvs! I guess in your case, you're using exclusively python3 'venv', but placing those at ~/.virtualenvs. I'm using primarily virtualenvwrapper, which by my configuration, places my NOT-A-'venv's at that same location. Doing it my way, invoking code with

$ workon my_env
(my_env) $ code .

the virtualenv isn't inherited at all. I think I can manually set my interpreter and restart code, but that's sort of cumbersome vs just bouncing around virtualenvs on the terminal.

On the other hand, if I make a 'venv' with

my-project-dir $ python3 -m venv my_project_venv
my-project-dir $ source my_project_venv/bin/activate
my-project-dir (my_project_venv) $ code .

code sees the venv directory inside my current project dir and uses that environment by default.

So anyway, if you're using python3 venv, just store the venv in your project directory and it seems vscode likes that. But if you're trying to use virtualenv/virtualenvwrapper, or else just need to keep your 'venv's in a separate dir, I'm not sure how to make either of those work right.

[davidthewatson]

Thanks @alexwhittemore. I can now start code from the command line. That's progress.

However, I use virtualenvwrapper with pyenv: pyenv local 3.6.2 then mkvirtualenv whatever From that point forward, workon whatever activates 3.6.2 with the package config from ~/.virtualenvs/ which, AFAIK, is by design.

I'd rather not store the virtualenvs in the project directory just to accommodate vscode. Instead, I'd prefer to figure out the bug where (according to the documentation) the various configs I've reported above should work.

[brettcannon]

@davidthewatson venvFolders is undocumented for a reason; it isn't meant to be used and we are planning on merging it with venvPath to make venvPaths (#1479 ). Basically @d3r3kk did a bad thing by even mentioning it 😉

Now in your ~/.virtualenvs, are there just directories of virtual environments, or are they somehow organized so that the virtual environments are another directory a level deeper? IOW what is ls ~/.virtualenvs and what does one of those directories contain? And have you tried using an absolute path for venvPath (we just had a bug report about us not expanding ~ in another setting so that might be an issue)?

And #118 is tracking the idea of supporting WORKON_HOME automatically.

@alexwhittemore you want #2124 for us to automatically use an activated virtual environment (we do automatically detect and use virtual environments that are in a top-level directory of your workspace folder).

[alexwhittemore]

@davidthewatson Ahhh - you're doing something I'm not used to (and don't understand) FWIW, my workflow for making python3 virtualenvs is mkvirtualenv -p python3 my_env_name where python3 is the global python 3.x.x from homebrew. I'm not sure what you get by doing pyenv local 3.6.2, and really, the whole differentiation between venv and what virtualenv does is a little beyond me, since they appear so similar.

And yes, I agree - storing environments in the project dir seems like the wrong approach to me for lots of reasons, but fwiw, it SEEMS to be what python's venv system assumes - maybe for just for simplicity in the venv documentation, maybe for best practice, I'm not sure.

@brettcannon That does look like what I'm after! I'll pursue my own parts of this discussion on those issues, thanks!

[brettcannon]

@alexwhittemore virtualenv predates venv as the external tool that provided virtual environments to Python. venv is the stdlib solution to the same problem to make it work more smoothly and proper integrate it into Python instead of virtualenv having to hack around stuff. 😉

And the pyenv local is using pyenv to specify what python3 should point to in the shell through pyenv.

And putting the virtual environment in the project is very common because you are expected to have an environment per project, so the thinking is why put it anywhere else?

[alexwhittemore]

And putting the virtual environment in the project is very common because you are expected to have an environment per project, so the thinking is why put it anywhere else?

That was my impression, but I think the logic there breaks down on discipline. (Not to argue, just musing:) That makes perfect sense to someone doing one-off projects or working locally, or whatever, but probably a lot less so to someone sticking to some kind of git workflow. My personal feeling is, absent git, yeah, why would virtual environment files live anywhere else? But if my project is in git, I REALLY prefer to avoid adding anything to .gitignore that's not strictly necessary - the pip list to rebuild my virtualenv should live there with the code, but definitely not all the extraneous stuff wasting space in my repo.

I feel like that's where the logic splits.

[davidthewatson]

The .virtualenvs directory in the user's home is the canonical design for virtualenvwrapper AFAICT:

http:https://virtualenvwrapper.readthedocs.io/en/latest/install.html#shell-startup-file

One of mine looks as you'd expect:

$ ls ~/.virtualenvs/litecure/
bin			etc			include			lib			pip-selfcheck.json	share

@brettcannon I tried changing the venvPath from the relative to the absolute in user settings:

{
    "python.venvPath": "/Users/davidwatson/.virtualenvs"
}

This had no effect. The virtualenv is neither prompted nor selected when starting the debugger.

[davidthewatson]

FWIW, I don't store my virtualenvs in project directories because IMO project directories should be specifically non-vendor content and vendor content should be clearly marked vendor. This mostly facilitates grepping and indexing without filtering mass quantities of unwanted noise. Trust me, I've worked on projects where someone committed the entire virtualenv into git and I'm sure you can imagine the heartache. :)

[alexwhittemore]

The .virtualenvs directory in the user's home is the canonical design for virtualenvwrapper AFAICT:
Yes, I thought I'd picked up that pattern from somewhere logical and mainstream.

Agree about leaving virtualenvs in the project directory. On the one hand I totally get the "well where else" argument, but I don't know why Python decided with venv to do it that way, since, yeah, SCM nightmare.

[brettcannon]

@alexwhittemore everyone simply ignores the directory, e.g. .gitignore containing .venv is default on GitHub.