-
-
Notifications
You must be signed in to change notification settings - Fork 18.5k
ENH: Add calamite engine to read_excel
#50581
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 6 commits
30da9a4
a47d3fb
6c1dd87
fd06ad9
8b6200a
6a8d822
efcb2fc
e1105de
6b50e0c
0784733
5971199
655318b
cc049cf
038133e
52c2cbd
2dc5e02
2076e11
6b0a7ac
a614089
256f9f9
eee8b4e
9fc2209
cf1268a
bebfec5
9019904
08a5616
677a224
8c55e5d
d817999
12aaf19
255e8fb
500fa9f
5d94728
15874c3
89ae49e
33e5b7e
85d31ec
a0d4193
a6b6fb2
0a431c5
745cd09
942a16a
8803ca9
2f5ffba
b8b1a9a
f5ab40d
02c2e7f
74a3e70
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -55,3 +55,6 @@ dependencies: | |
- xlrd | ||
- xlsxwriter | ||
- zstandard | ||
|
||
- pip: | ||
- python-calamine |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -69,3 +69,6 @@ dependencies: | |
- pandas-gbq | ||
- pyyaml | ||
- py | ||
|
||
- pip: | ||
- python-calamine |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -61,3 +61,4 @@ dependencies: | |
|
||
- pip: | ||
- pyqt5==5.15.1 | ||
- python-calamine==0.0.5 |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -55,3 +55,6 @@ dependencies: | |
- xlrd | ||
- xlsxwriter | ||
- zstandard | ||
|
||
- pip: | ||
- python-calamine |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -55,3 +55,6 @@ dependencies: | |
- xlrd | ||
- xlsxwriter | ||
- zstandard | ||
|
||
- pip: | ||
- python-calamine |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -106,7 +106,7 @@ Other enhancements | |
- Improved error message in :func:`to_datetime` for non-ISO8601 formats, informing users about the position of the first error (:issue:`50361`) | ||
- Improved error message when trying to align :class:`DataFrame` objects (for example, in :func:`DataFrame.compare`) to clarify that "identically labelled" refers to both index and columns (:issue:`50083`) | ||
- Performance improvement in :func:`to_datetime` when format is given or can be inferred (:issue:`50465`) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We are in the processing of cutting 2.0.0 now; at this point should target 2.1.0 for this PR There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @kostyafarber, hi! Can you merge main in issue-50395? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yep will do There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @dimastbk merged! There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Looks like this is still in the v2.0.0 whatsnew There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, I asked for merging main in pr branch. Now fixed. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. merged. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You should revert any changes to this file - the v2.0.0.rst file shouldn't be touched as part of this PR |
||
- | ||
- Added ``calamite`` as an engine to ``read_excel`` (:issue: ``50395``) | ||
|
||
.. --------------------------------------------------------------------------- | ||
.. _whatsnew_200.notable_bug_fixes: | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -118,3 +118,4 @@ dependencies: | |
|
||
- pip: | ||
- sphinx-toggleprompt | ||
- python-calamine |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,69 @@ | ||
from __future__ import annotations | ||
|
||
from io import ( | ||
BufferedReader, | ||
BytesIO, | ||
) | ||
from pathlib import PurePath | ||
from tempfile import NamedTemporaryFile | ||
from typing import Any | ||
|
||
from python_calamine import ( | ||
get_sheet_data, | ||
get_sheet_names, | ||
) | ||
kostyafarber marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
from pandas.io.excel._base import ( | ||
BaseExcelReader, | ||
inspect_excel_format, | ||
) | ||
|
||
|
||
class __calamine__: | ||
pass | ||
|
||
|
||
class CalamineExcelReader(BaseExcelReader): | ||
book: str | ||
_sheet_names: list[str] | None = None | ||
|
||
@property | ||
def _workbook_class(self) -> type[__calamine__]: | ||
return __calamine__ | ||
|
||
def load_workbook( | ||
self, filepath_or_buffer: str | PurePath | BufferedReader | BytesIO | ||
) -> str: | ||
if isinstance(filepath_or_buffer, BufferedReader): | ||
filepath_or_buffer = filepath_or_buffer.name | ||
|
||
elif isinstance(filepath_or_buffer, BytesIO): | ||
ext = inspect_excel_format(filepath_or_buffer) | ||
with NamedTemporaryFile(suffix=f".{ext}", delete=False) as tmp_file: | ||
tmp_file.write(filepath_or_buffer.getvalue()) | ||
filepath_or_buffer = tmp_file.name | ||
|
||
elif isinstance(filepath_or_buffer, PurePath): | ||
filepath_or_buffer = filepath_or_buffer.as_posix() | ||
|
||
assert isinstance(filepath_or_buffer, str) | ||
|
||
self._sheet_names = get_sheet_names(filepath_or_buffer) | ||
return filepath_or_buffer | ||
|
||
@property | ||
def sheet_names(self) -> list[str]: | ||
if self._sheet_names is None: | ||
self._sheet_names = get_sheet_names(self.book) | ||
return self._sheet_names | ||
|
||
def get_sheet_by_name(self, name: str) -> int: | ||
self.raise_if_bad_sheet_by_name(name) | ||
return self.sheet_names.index(name) | ||
|
||
def get_sheet_by_index(self, index: int) -> int: | ||
self.raise_if_bad_sheet_by_index(index) | ||
return index | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I presume there is more being added here? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This was ported over from the implementation in the library's repo here, but obvs wasn't implemented by the maintainer. I'm unsure if this is something the maintainer of the library will add in the future but I can remove this for now. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fixed. |
||
|
||
def get_sheet_data(self, sheet: int, convert_float: bool) -> list[list[Any]]: | ||
return get_sheet_data(self.book, sheet) |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -87,4 +87,5 @@ feedparser | |
pyyaml | ||
requests | ||
sphinx-toggleprompt | ||
python-calamine | ||
setuptools>=61.0.0 |
Uh oh!
There was an error while loading. Please reload this page.