RFC: Python 2 support

[effigies]

As I'm sure we're all aware, CPython 2 support ends on January 1, 2020, and a number of projects have committed to cutting off Python 2 support in the coming year, some already having effected the change in master. In particular, numpy, scipy, scikit-learn and scikit-image, which nibabel either depends on or is often used in conjunction with, have all committed to ending their Python 2 support.

We cannot realistically extend our support of Python 2 much farther, even if we want to. I would like to propose the upcoming 2.4 series as the last series with any Python 2 support at all, and commit to bug-fixes in 2.4.x until January 2020. This is a short timeline, but hopefully piggy-backing on the zeitgeist will help keep it from being too jarring.

I am open to someone else maintaining the 2.4.x branch for a short while after January 2020, but I think it should be short.

Assuming we settle on this scheme, nibabel 3.0 will remove all Python 2 support, and I'll do my best to ensure that the PyPI metadata will require Python 3+, so that Python 2's pip will not attempt to install nibabel>=3.

@nipy/core @nipy/team-nibabel @GaelVaroquaux @pauldmccarthy @afni-rickr

[yarikoptic]

What will be the minimal python3 version you are aiming for, or in other words what feature(s) of python3 line you would like to start using. here we collected some notable new features with python3 versions where they were introduced. May be there is some nice concise overview already of a similar kind?

[effigies]

I think my overall feeling about nibabel is that we should be pretty conservative and support as far back as we can, but outlasting CPython itself is not a good use of effort. Python 3.4 is supposed to have a final release this month, which means it will predecease 2.7 by a little bit.

So my inclination would be to support 2.7 and 3.4 through nibabel 2.4.x, and jump to 3.5+ for nibabel 3.0. We won't get f-strings or data classes, but I don't see those as particularly critical to nibabel's development. The main thing I know of that 3.6+ would buy us that makes me look forward to dropping 3.5 is insertion-ordered dictionaries obviating the need for OrderedDict, but I think maintaining compatibility outweighs that desire.

[matthew-brett]

Sounds reasonable to me. I wonder whether we'll need Python 2 support (via 2.4) for a year or so after Python 2 end-of-life though. I doubt that would be a lot of extra work.

[effigies]

That's probably true. Mostly will mean needing to take the maint/* branches seriously.

[yarikoptic]

Do we need maint/ prefix? I would just have release branches like 2.3.x , 2.4.x

[effigies]

It's what we've been doing. At least for 2.4.x, let's continue with this trend, following the principle of least surprise. I don't have a strong commitment to the form, though, so if the population vector points in that direction, I'm okay with dropping the maint/ prefix at 3.0.x.

[yarikoptic]

Ah right, missed that it already exists - all good then, not worth disturbing the force

[effigies]

Oh, minor amendment: Forgot that by the schedule put forward in #734, 2.5 will be due out in June (give-or-take). So that would be the final Python 2-supporting series.

[raamana]

Do you guys think knowing the usage extent of nibabel with python < 3.0 would help? If knowing that (and why) would help this decision making process, I think pypinfo can help us here: https://github.com/ofek/pypinfo

Chris, regarding dict in python 3.5, please note:

"The order-preserving aspect of this new implementation is considered an implementation detail and should not be relied upon (this may change in the future, but it is desired to have this new dict implementation in the language for a few releases before changing the language spec to mandate order-preserving semantics for all current and future Python implementations; this also helps preserve backwards-compatibility with older versions of the language where random iteration order is still in effect, e.g. Python 3.5)."

https://docs.python.org/3.6/whatsnew/3.6.html#new-dict-implementation

[effigies]

I don't know the usage extent, but I believe that PyMVPA depends on nibabel and is still Python 2-only.

[raamana]

I just quickly ran pypinfo, and here are the download stats for nibabel for the last 800 days (~2 years). I think there is some noise in these numbers (and some repeat downloads I believe), but the their fractions should help us give a rough idea of usage. So python 2.7 and python 2.6 account for about 34% usage.

$ 14:08:47 miner ~ >>  pypinfo -sd -800 nibabel pyversion
Served from cache: False
Data processed: 589.72 GiB
Data billed: 589.72 GiB
Estimated cost: $2.88

| python_version | download_count |
| -------------- | -------------- |
| 3.6            |        362,950 |
| 2.7            |        243,117 |
| 3.5            |         70,889 |
| 3.4            |         21,275 |
| 3.7            |         14,280 |
| 2.6            |            360 |
| 3.3            |             96 |
| 3.2            |             21 |
| 3.8            |              6 |
| None           |              3 |
| Total          |        712,997 |

$ 14:10:26 miner ~ >>  

[yarikoptic]

It might be just that they are just still tested against and instated using pip. And there would be a sudden drop as those projects move on ;-) I really doubt that any recent 2.6 download had a human behind it. 2.7 though IMHO will still see use for a year or two on systems where 2.7 is the default one (majority of released Linux distributions ATM? ;-))

[raamana]

True - it is possible a non-trivial portion of the downloads for python 2.7 are from CI testing matrices. That doesn't necessarily imply they are also tested for and support python 3+.

It would have been awesome if we had a way to mine the pypi index to find out all the packages that declared nibabel as a dependency, and see how many we would break by orphaning python 2.7. Usually this link shows it, but perhaps only admins can see it.

[effigies]

Usually this link shows it, but perhaps only admins can see it.

No, GitHub's not very good about following Python dependency graphs. Many actual dependencies will fail to be detected.