Add PYTHONPATH to environment variables if the Python extension makes it available

[mattseddon]

Closes #1791

This PR uses the ms-python.python extension's new API to grab their PYTHONPATH variable and add it to the environment variables used to create DVC processes. In order for users to set this up they need to add a file containing PYTHONPATH=. This can be a .env file at the root of the workspace (default for the extension) or they need to set the python.envFile configuration option.

Demo (fixes issue in magnetic-tiles-defect)

Screen.Recording.2023-06-06.at.4.53.48.pm.mov

[codeclimate]

Identical blocks of code found in 2 locations. Consider refactoring.

[codeclimate]

Identical blocks of code found in 2 locations. Consider refactoring.

[julieg18]

Add PYTHONPATH to environment variables if the Python extension makes it available

Is PYTHONPATH moved to stable or is it still deprecated?

[shcheklein]

In order for users to set this up they need to add a file containing PYTHONPATH=. This can be a .env file at the root of the workspace (default for the extension) or they need to set the python.envFile configuration option.

I'm still lost on why PYTHONPATH is required. #1791 seems to be broader - it's about other envs, other variables, etc. Unless I'm missing something.

[mattseddon]

Add PYTHONPATH to environment variables if the Python extension makes it available

Is PYTHONPATH moved to stable or is it still deprecated?

It has been moved to stable:

• https://github.com/microsoft/vscode-python/blob/main/src/client/apiTypes.ts#L60
• https://github.com/microsoft/vscode-python/wiki/Python-Environment-APIs

[mattseddon]

In order for users to set this up they need to add a file containing PYTHONPATH=. This can be a .env file at the root of the workspace (default for the extension) or they need to set the python.envFile configuration option.

I'm still lost on why PYTHONPATH is required. #1791 seems to be broader - it's about other envs, other variables, etc. Unless I'm missing something.

#1815 is the ticket that spawned #1791. Under #1791 two separate users detailed the same issue (here and here). They want to use relative imports for the modules within their project. By using this API we will respect whatever they've set for PYTHONPATH through either a .env file or the terminal.integrated.env.* settings (details here: https://code.visualstudio.com/docs/python/environments#_use-of-the-pythonpath-variable). Outside of that, no one has really complained about environment activation which leads me to think that this is a good enough solution to close the ticket.

[mattseddon]

We can leave the ticket open but this is still useful IMO.

[codeclimate]

Code Climate has analyzed commit f401d36 and detected 2 issues on this pull request.

Here's the issue category breakdown:

Category	Count
Duplication	2

The test coverage on the diff in this pull request is 96.7% (85% is the threshold).

This pull request will bring the total coverage in the repository to 95.2% (0.0% change).

View more on Code Climate.