Please stop relying on xlrd

[cjw296]

Problem description

Guys, xlrd is unmaintained. I know, I'm the maintainer!

Please can you use openpyxl for reading xlsx files instead.

If you really must keep support for the ancient and outdated .xls format, by all means fall back to xlrd, but warn people they're on shaky territory and should sort their data source out!

[jorisvandenbossche]

Thanks for the heads up!
There is an open issue about adding reading support based on openpyxl: #11499, and there is an active PR working on it: #25092

[WillAyd]

Closing as a duplicate of the aforementioned issue

[jorisvandenbossche]

@WillAyd one thing we should consider, given the above, is actually change the default (which I think is not yet discussed in the other issue / PR? As it is only about adding another engine that you can specify with engine='openpyxl' ?)

[WillAyd]

Yea for sure - I imagine not something we want to do at the outset though no?

…

Sent from my iPhone

On May 26, 2019, at 5:50 AM, Joris Van den Bossche ***@***.***> wrote: @WillAyd one thing we should consider, given the above, is actually change the default (which I think is not yet discussed in the other issue / PR? As it is only about adding another engine that you can specify with engine='openpyxl' ?) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[jorisvandenbossche]

Not sure. The idea could be that let's first enable it so people can specify engine='openpyxl'. But how many people will do that? We might only get feedback when actually changing the default (at least for xlsx)

[jorisvandenbossche]

(but for sure, the first important thing is getting actual openpyxl support in)

[jreback]

we don’t want to rely going forward on a deprecated library but i don’t think anything is broken except for a very small edge case (and that’s just a warning)?

[WillAyd]

Yes that’s the only issue as of now

…

Sent from my iPhone

On May 26, 2019, at 7:14 AM, Jeff Reback ***@***.***> wrote: we don’t want to rely going forward on a deprecated library but i don’t think anything is broken except for a very small edge case (and that’s just a warning)? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[cjw296]

So, a bit of background on this from an xlrd maintainer: it’s not a fun code base to work on (can’t really unit test, etc) and whether a change is safe to make is fiendishly difficult to know for all cases. So, no one has really stepped up to fill the void that myself and John left. The library is stable, and the problems reported have either been DeprecationWarnings (wow, what a waste of time...) or edge cases where people are feeding in invalid files and expecting them to parse because some other library or program makes allowances for them. My take is that openpyxl absolutely should be the default. For xlsx it’s going to be an infinitely better choice. For xls, I’m not aware of a choice other than xlrd, so it’ll likely have to do. But, please can you find a way of explaining that invalid xls files churned out by some lab machine or website should not be expected to parse. If there’s any way for you guys to have those issues logged against pandas and not xlrd, that would make life a lot happier. Of course, if you guys want to take up maintenance of xlrd, that would be an interesting conversation, drop me an email!