Optional interpreters

[snoack]

For projects which support a wide range of Python versions, it seems an unnecessary hurdle to force every contributor to install each Python version locally, in particular if not all versions can be installed through the system's package manager, in particular if the tests run additionally by the CI anyway.

It seems this use case has already been considered with the skip_missing_interpreter option. However, I'm not feeling comfortable making each interpreter optional. For example, if code is meant to be compatible with Python 2.7 and 3.4+, everybody should always test on Python 2.7 and 3.4, ideally as well on Python 3.5 and 3.6 but if those interpreters aren't installed it is acceptable to skip those environments.

Some suggestions, how tox could account for that scenario:

1. Allow skip_missing_interpreter to be used per testenv.
2. Allow skip_missing_interpreter to list particular interpreters that might be skipped.
3. Add syntax to indicate environments as optional, e.g. envlist = py27,py34,py35?,py36?.

[obestwalter]

@snoack sounds like a good idea to me.

[hpk42]

@snoack good point. It seems i did not pay close attention when this was added. Usually i prefer all new settings to be per test environment. Not sure if it's a good idea now to introduce the same config option to the test environment section now, might be confusing. extending the envlist syntax is probably too cumbersome as it already has special syntax for generative environments. Extending skip_missing_interpreters to allow specifying a list of environments is also a little bit tricky because it's currently a boolean but "yes", "no", "true", "false" is all allowed IIRC. But maybe that's feasible -- @snoack feel invited to try a PR implementing it. @obestwalter is planning a release for tomorrow btw :)

[obestwalter]

OT: @hpk42 about that release ... I am down with the flu atm so this will have to wait a few more days I am afraid. Can't help checking the issues here now and then though :)

[nicoddemus]

A quick comment: although I agree that extending the envlist syntax might be a little cumbersome, it has the advantage of making it natural to allow users to make some test environments optional, for example all variants of py33 and trial envtests on a complex generative environment list. The added flexibility might be worth it.

[hpk42]

@nicoddemus do you have precise syntax suggestions that don't conflict with what we have? So far envlist kind of follows, roughly uses bash pattern syntax, not sure there is anything we could leverage.

[nicoddemus]

Unfortunately I don't have any suggestions for a precise syntax right now. 😑

[obestwalter]

I just opened #600 and feel like this one would be good to be in the new config format instead of oveburdening the old format even more (if this actually ever gets done).

[gaborbernat]

I agree we should support this.