Skip to content

BUG: NDArrayBacked2DTests compliance tests from pandas are failing #65

Closed
@tswast

Description

@tswast
(pip-3.10-prerelease) ➜  python-db-dtypes-pandas git:(issue28-NDArrayBacked2DTests) ✗ pytest tests/unit/test_date_compliance.py
======================================================= test session starts ========================================================
platform darwin -- Python 3.10.0, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /Users/swast/src/github.com/googleapis/python-db-dtypes-pandas
collected 32 items                                                                                                                 

tests/unit/test_date_compliance.py ....................F.....F.....                                                          [100%]

============================================================= FAILURES =============================================================
__________________________________________ Test2DCompat.test_reductions_2d_axis0[median] ___________________________________________

self = <test_date_compliance.Test2DCompat object at 0x11ec56740>
data = <DateArray>
[  datetime.date(1900, 1, 1),  datetime.date(1900, 1, 14),
  datetime.date(1900, 1, 27),   datetime.date(1...1),  datetime.date(2099, 12, 4),
 datetime.date(2099, 12, 17), datetime.date(2099, 12, 30)]
Length: 5620, dtype: dbdate
method = 'median'

    @pytest.mark.parametrize("method", ["mean", "median", "var", "std", "sum", "prod"])
    def test_reductions_2d_axis0(self, data, method):
        arr2d = data.reshape(1, -1)
    
        kwargs = {}
        if method == "std":
            # pass ddof=0 so we get all-zero std instead of all-NA std
            kwargs["ddof"] = 0
    
        try:
>           result = getattr(arr2d, method)(axis=0, **kwargs)

/usr/local/Caskroom/miniconda/base/envs/pip-3.10-prerelease/lib/python3.10/site-packages/pandas/tests/extension/base/dim2.py:194: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <DateArray>
[
[  datetime.date(1900, 1, 1),  datetime.date(1900, 1, 14),
  datetime.date(1900, 1, 27),   datetime.date...atetime.date(2099, 12, 4),
 datetime.date(2099, 12, 17), datetime.date(2099, 12, 30)]
]
Shape: (1, 5620), dtype: dbdate

    def median(
        self,
        *,
        axis: Optional[int] = None,
        out=None,
        overwrite_input: bool = False,
        keepdims: bool = False,
        skipna: bool = True,
    ):
        pandas_backports.numpy_validate_median(
            (),
            {"out": out, "overwrite_input": overwrite_input, "keepdims": keepdims},
        )
        result = pandas_backports.nanmedian(self._ndarray, axis=axis, skipna=skipna)
>       return self._box_func(result)

db_dtypes/core.py:163: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <DateArray>
[
[  datetime.date(1900, 1, 1),  datetime.date(1900, 1, 14),
  datetime.date(1900, 1, 27),   datetime.date...atetime.date(2099, 12, 4),
 datetime.date(2099, 12, 17), datetime.date(2099, 12, 30)]
]
Shape: (1, 5620), dtype: dbdate
x = array(['1900-01-01T00:00:00.000000000', '1900-01-14T00:00:00.000000000',
       '1900-01-27T00:00:00.000000000', ...,
...T00:00:00.000000000', '2099-12-17T00:00:00.000000000',
       '2099-12-30T00:00:00.000000000'], dtype='datetime64[ns]')

    def _box_func(self, x):
>       if pandas.isnull(x):
E       ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

db_dtypes/__init__.py:253: ValueError

During handling of the above exception, another exception occurred:

self = <test_date_compliance.Test2DCompat object at 0x11ec56740>
data = <DateArray>
[  datetime.date(1900, 1, 1),  datetime.date(1900, 1, 14),
  datetime.date(1900, 1, 27),   datetime.date(1...1),  datetime.date(2099, 12, 4),
 datetime.date(2099, 12, 17), datetime.date(2099, 12, 30)]
Length: 5620, dtype: dbdate
method = 'median'

    @pytest.mark.parametrize("method", ["mean", "median", "var", "std", "sum", "prod"])
    def test_reductions_2d_axis0(self, data, method):
        arr2d = data.reshape(1, -1)
    
        kwargs = {}
        if method == "std":
            # pass ddof=0 so we get all-zero std instead of all-NA std
            kwargs["ddof"] = 0
    
        try:
            result = getattr(arr2d, method)(axis=0, **kwargs)
        except Exception as err:
            try:
                getattr(data, method)()
            except Exception as err2:
                assert type(err) == type(err2)
                return
            else:
>               raise AssertionError("Both reductions should raise or neither")
E               AssertionError: Both reductions should raise or neither

/usr/local/Caskroom/miniconda/base/envs/pip-3.10-prerelease/lib/python3.10/site-packages/pandas/tests/extension/base/dim2.py:202: AssertionError
__________________________________________ Test2DCompat.test_reductions_2d_axis1[median] ___________________________________________

self = <DateArray>
[
[  datetime.date(1900, 1, 1),  datetime.date(1900, 1, 14),
  datetime.date(1900, 1, 27),   datetime.date...atetime.date(2099, 12, 4),
 datetime.date(2099, 12, 17), datetime.date(2099, 12, 30)]
]
Shape: (1, 5620), dtype: dbdate
x = array(['1999-12-31T12:00:00.000000000'], dtype='datetime64[ns]')

    def _box_func(self, x):
        if pandas.isnull(x):
            return None
        try:
>           return x.astype("<M8[us]").astype(datetime.datetime).date()
E           AttributeError: 'numpy.ndarray' object has no attribute 'date'

db_dtypes/__init__.py:256: AttributeError

During handling of the above exception, another exception occurred:

self = <test_date_compliance.Test2DCompat object at 0x11f1b3be0>
data = <DateArray>
[  datetime.date(1900, 1, 1),  datetime.date(1900, 1, 14),
  datetime.date(1900, 1, 27),   datetime.date(1...1),  datetime.date(2099, 12, 4),
 datetime.date(2099, 12, 17), datetime.date(2099, 12, 30)]
Length: 5620, dtype: dbdate
method = 'median'

    @pytest.mark.parametrize("method", ["mean", "median", "var", "std", "sum", "prod"])
    def test_reductions_2d_axis1(self, data, method):
        arr2d = data.reshape(1, -1)
    
        try:
>           result = getattr(arr2d, method)(axis=1)

/usr/local/Caskroom/miniconda/base/envs/pip-3.10-prerelease/lib/python3.10/site-packages/pandas/tests/extension/base/dim2.py:239: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <DateArray>
[
[  datetime.date(1900, 1, 1),  datetime.date(1900, 1, 14),
  datetime.date(1900, 1, 27),   datetime.date...atetime.date(2099, 12, 4),
 datetime.date(2099, 12, 17), datetime.date(2099, 12, 30)]
]
Shape: (1, 5620), dtype: dbdate

    def median(
        self,
        *,
        axis: Optional[int] = None,
        out=None,
        overwrite_input: bool = False,
        keepdims: bool = False,
        skipna: bool = True,
    ):
        pandas_backports.numpy_validate_median(
            (),
            {"out": out, "overwrite_input": overwrite_input, "keepdims": keepdims},
        )
        result = pandas_backports.nanmedian(self._ndarray, axis=axis, skipna=skipna)
>       return self._box_func(result)

db_dtypes/core.py:163: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <DateArray>
[
[  datetime.date(1900, 1, 1),  datetime.date(1900, 1, 14),
  datetime.date(1900, 1, 27),   datetime.date...atetime.date(2099, 12, 4),
 datetime.date(2099, 12, 17), datetime.date(2099, 12, 30)]
]
Shape: (1, 5620), dtype: dbdate
x = array(['1999-12-31T12:00:00.000000000'], dtype='datetime64[ns]')

    def _box_func(self, x):
        if pandas.isnull(x):
            return None
        try:
            return x.astype("<M8[us]").astype(datetime.datetime).date()
        except AttributeError:
>           x = numpy.datetime64(x)
E           ValueError: Could not convert object to NumPy datetime

db_dtypes/__init__.py:258: ValueError

During handling of the above exception, another exception occurred:

self = <test_date_compliance.Test2DCompat object at 0x11f1b3be0>
data = <DateArray>
[  datetime.date(1900, 1, 1),  datetime.date(1900, 1, 14),
  datetime.date(1900, 1, 27),   datetime.date(1...1),  datetime.date(2099, 12, 4),
 datetime.date(2099, 12, 17), datetime.date(2099, 12, 30)]
Length: 5620, dtype: dbdate
method = 'median'

    @pytest.mark.parametrize("method", ["mean", "median", "var", "std", "sum", "prod"])
    def test_reductions_2d_axis1(self, data, method):
        arr2d = data.reshape(1, -1)
    
        try:
            result = getattr(arr2d, method)(axis=1)
        except Exception as err:
            try:
                getattr(data, method)()
            except Exception as err2:
                assert type(err) == type(err2)
                return
            else:
>               raise AssertionError("Both reductions should raise or neither")
E               AssertionError: Both reductions should raise or neither

/usr/local/Caskroom/miniconda/base/envs/pip-3.10-prerelease/lib/python3.10/site-packages/pandas/tests/extension/base/dim2.py:247: AssertionError
===================================================== short test summary info ======================================================
FAILED tests/unit/test_date_compliance.py::Test2DCompat::test_reductions_2d_axis0[median] - AssertionError: Both reductions shoul...
FAILED tests/unit/test_date_compliance.py::Test2DCompat::test_reductions_2d_axis1[median] - AssertionError: Both reductions shoul...
=================================================== 2 failed, 30 passed in 2.31s ===================================================

Metadata

Metadata

Assignees

Labels

api: bigqueryIssues related to the googleapis/python-db-dtypes-pandas API.priority: p2Moderately-important priority. Fix may not be included in next release.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions