Backward compatibility and numpy 2.0

[mscheltienne]

MNE seems to support numpy 2.0 as of version 1.6. Now, if we follow SPEC0, downstream packages should support their core dependencies (e.g. MNE) during 2 years. Yet, if you do pip install mne==1.4.2, you end up with a broken installation because there is no upper pin on the numpy version.

Should we add an upper version pin <2 on numpy on maint/1.2, maint/1.3, maint/1.4, maint/1.5, and cut a bugfix release on those MNE versions?
I'm not sure if this is the best solution to ensure easy support for downstream packages, any other ideas?

For x-ref, CIs failing on mne-lsl: mne-tools/mne-lsl#289
Might maybe also impact other packages in the ecosystem?

[larsoner]

Our policy has always been to only support one version back and no father. There are probably other breakages with other packages in 1.2 for example and it'll increase maintenance burden (potentially a lot) with little gain for end users I think to continuously monitor and fix those. So I'd rather continue not to update anything father back than 1.7.

[larsoner]

Also I think that might be a misreading of SPEC0. I think it is meant to say that a MNE released today should work with latest NumPy back to the one from two years ago, not that MNE released version from two years ago should be continuously updated to work with NumPy from today.

[drammock]

Our policy has always been to only support one version back and no father

agree, and this is documented here: https://github.com/mne-tools/mne-python/blob/main/SECURITY.md (not super findable, but still)

I think that might be a misreading of SPEC0. I think it is meant to say that a MNE released today should work with latest NumPy back to the one from two years ago, not that MNE released version from two years ago should be continuously updated to work with NumPy from today.

agree.

[mscheltienne]

Also I think that might be a misreading of SPEC0. I think it is meant to say that a MNE released today should work with latest NumPy back to the one from two years ago, not that MNE released version from two years ago should be continuously updated to work with NumPy from today.

I completely agree with that, but from the point of view of a downstream package, it means that it should work with an MNE released today or 2 years ago. Yet an install of MNE from 2 years ago yields a broken install nowadays. What would be the fix for a downstream package then?

(1) ignore the issue, it's up to the user to know that if he runs mne 1.4.2 he can not run numpy 2.0
(2) ...? I don't have a (2), pinning to mne 1.6 or above is too much of a restriction.

[mscheltienne]

There are probably other breakages with other packages in 1.2 for example and it'll increase maintenance burden (potentially a lot) with little gain for end users I think to continuously monitor and fix those.

Agree as well, but numpy 2.0 might be kind of an exception as to how 'big' a breakage it is.

[drammock]

from the point of view of a downstream package, it means that it should work with an MNE released today or 2 years ago. Yet an install of MNE from 2 years ago yields a broken install

ah OK, I see the dilemma now. If you want MNE-LSL to follow SPEC-0, then you either need to

1. pin numpy yourself to whatever is MNE-1.4 compatible, or
2. backport those pins on old MNE versions (as you suggested)
3. pin MNE-LSL to use MNE>=1.6 (which you say is unacceptable?)

I don't want to "open the floodgates" on keeping old MNE versions patched, but indeed it would have been better to have had upper (major-version) pins back then.

So now the question is whether (1) or (2) is better/easier. I doubt MNE-LSL is the only downstream affected by this (just the only one telling us so far 😅), so I guess I'm +0.5 on actually backporting the pins.

[larsoner]

pin numpy yourself to whatever is MNE-1.4 compatible, or

I think this is fine. If you really want to use an old version of any library you're going to face issues like this. I don't think major libraries are planning to retroactively pin -- looking at a pandas release 1.5.x series from ~22 months ago for example:

>>> import requests
>>> out = requests.get("https://pypi.org/pypi/pandas/1.5.3/json").json()
>>> [ent for ent in out['info']['requires_dist'] if "numpy" in ent]
['numpy (>=1.20.3) ; python_version < "3.10"', 'numpy (>=1.21.0) ; python_version >= "3.10"', 'numpy (>=1.23.2) ; python_version >= "3.11"']

I also vaguely recall somewhere in the (long!) NumPy 2.0 discussions people mentioning how trying to retroactively pin like this ends up being not super helpful, because even you cut a 1.4.3 with a pin, 1.4.2 doesn't (and never will) so you can still end up with a broken env -- for one example, pip install mne==1.4 "numpy>2" will still work, it will just install (the broken) MNE 1.4.2.

Going forward we could consider a numpy<3 in our pyproject.toml for example -- I think they recommend something like this officially.

[mscheltienne]

OK, that's convincing. +1 on numpy<3 in pyproject.toml, and indeed pinning ourself to whatever is MNE 1.4 compatible sounds like the best approach.

[drammock]

OK! So NumPy pins go in the downstreams. By the way @larsoner should we add MNE to numpy/numpy#26191 (ecosystem compatibility tracker)?

[larsoner]

Sure -- I was going to say we need to look back to figure out the min version that's np2 compatible, but we could just say 1.7.0 since 1.6.x and older are unsupported anyway.