COMPAT: Consider dropping numpy 1.6 support

[cpcloud]

NumPy 1.6

NumPy 1.6 introduced numerous bugs around datetimes, forcing a plethora of hacks, and causing numerous stress-induced ulcers for an uncountably infinite number of people.

The time for pandas to lose this vestigial organ has come. The main reason for this is that datetime support is just broken, in nearly every sense of the word. Take a look at some of the conversion code in pandas/core/common.py and you'll see what I'm talking about.

I think we've probably reached a point where supporting numpy 1.6 is just going to cause more pain than it's worth. To me it makes no sense to want to keep upgrading pandas while keeping numpy <= 1.6.1. If you want modern pandas, you need to use modern-ish versions of its dependencies.

[jreback]

i think i raised this for #5108, but maybe its time

[jreback]

cc @yarikoptic

can you give an indication when major distros moving to 1.7 for their stable? (iirc somewhere jessie is doing this in oct of this year?)

[jtratner]

To me it makes no sense to want to keep upgrading pandas while keeping numpy <= 1.6.1. If you want modern pandas, you need to use modern-ish versions of its dependencies.

Excellent point. 👍

[yarikoptic]

here is the current state of numpy:

Ubuntu   1:1.3.0-3build1    http://packages.ubuntu.com/lucid/python-numpy
Ubuntu   1:1.6.1-6ubuntu1   http://packages.ubuntu.com/precise/python-numpy
Ubuntu   1:1.7.1-1ubuntu1   http://packages.ubuntu.com/raring/python-numpy
Ubuntu   1:1.7.1-1ubuntu1   http://packages.ubuntu.com/saucy/python-numpy
Ubuntu   1:1.8.1-1ubuntu1   http://packages.ubuntu.com/trusty/python-numpy
Ubuntu   1:1.8.1-1ubuntu1   http://packages.ubuntu.com/utopic/python-numpy
Debian   1:1.4.1-5          http://packages.debian.org/squeeze/python-numpy
Debian   1:1.6.2-1.2        http://packages.debian.org/wheezy/python-numpy
Debian   1:1.8.1-1+b1       http://packages.debian.org/jessie/python-numpy
Debian   1:1.8.1-1+b2       http://packages.debian.org/sid/python-numpy

so jessie will have the "modern" numpy, but current stable is at 1.6.2 and current pandas still happy on it. It would be great if dropping it could may be postponed after the next pandas release? according to reasoning above current release has broken handling of datetime on wheezy with numpy 1.6.2 -- then "dropping" support of numpy 1.6 in the next release would simply leave people on stable debian with a broken pandas, or have I misread it?

[jreback]

@yarikoptic you have misread.

pandas 'fixes' the timedelta support on < 1.7 numpy (using pandas operations). even though numpy is hopelessly broken (its much better in 1.8/1.9). In reality we now DO raise on < 1.7 numpy for unsupported ops.

the issue is that we have lots of test that are skipped on numpy < 1.7 and a fair number of ugly work-arounds for these.

I actually don't see dropping support for < 1.7 as much of a problem, because this is ONLY an issue if:

• you have a fairly old build (i use wheezy so am guilty!)
• you install/upgrade to current pandas using PIP
• you don't install numpy but instead use the system installed version

why ANYONE would do this is beyond me. If you are going to install pandas latest, what's the big deal with install numpy current?

[jreback]

@cpcloud let's separate the 2.6 part of this to a new issue

[dsm054]

Now that Python has officially blessed virtual environments with the adoption of venv, the canonical response to "but my system only has library version X installed" is now "so?"

+1 in favour of dropping broken (for our use cases) numpy.

[jreback]

@cpcloud can you put this on the mailing list as well, just to see if we are missing anything here. (and copy numpy-discussion too).

[cpcloud]

Sure

[yarikoptic]

@jreback regarding three bullet points. "you install/upgrade to current pandas using PIP" -- I agree that if someone does it, he is on his own ;) I was primarily concerned to have a reasonably good version within NeuroDebian for wheezy for at least another year (so people have time to upgrade from wheezy to jessie). If you think that 0.14.0+git17-g3849d5d-1~nd70+1 should be fine -- then go ahead with dropping the support. If you feel that it might be worth making a quick point or snapshot release before you drop 1.6.2 numpy support -- might even be better ;-)

[jreback]

@yarikoptic we are planning on releasing 0.14.1 tomorrow, which includes < 1.7 support. @cpcloud mentioned this for 0.15.0 approx due end of sept 2014. That work?