Merge branch 'master' of https://github.com/pandas-dev/pandas into libtsmore

Author: jbrockmendel

ci/travis-27.yaml

@@ -35,7 +35,7 @@ dependencies:
   - s3fs
   - scipy
   - sqlalchemy=0.9.6
-  - xarray=0.8.0
+  - xarray=0.9.6
   - xlrd=0.9.2
   - xlsxwriter=0.5.2
   - xlwt=0.7.5

doc/source/api.rst

@@ -515,7 +515,6 @@ Reshaping, sorting
    Series.repeat
    Series.squeeze
    Series.view
-   Series.sortlevel
 
 
 Combining / joining / merging

doc/source/whatsnew/v0.24.0.txt

@@ -201,7 +201,6 @@ Other Enhancements
 - :func:`~DataFrame.to_csv`, :func:`~Series.to_csv`, :func:`~DataFrame.to_json`, and :func:`~Series.to_json` now support ``compression='infer'`` to infer compression based on filename extension (:issue:`15008`).
   The default compression for ``to_csv``, ``to_json``, and ``to_pickle`` methods has been updated to ``'infer'`` (:issue:`22004`).
 - :func:`to_timedelta` now supports iso-formated timedelta strings (:issue:`21877`)
-- Comparing :class:`Timedelta` with unknown types now return ``NotImplemented`` instead of ``False`` (:issue:`20829`)
 - :class:`Series` and :class:`DataFrame` now support :class:`Iterable` in constructor (:issue:`2193`)
 - :class:`DatetimeIndex` gained :attr:`DatetimeIndex.timetz` attribute. Returns local time with timezone information. (:issue:`21358`)
 - :meth:`round`, :meth:`ceil`, and meth:`floor` for :class:`DatetimeIndex` and :class:`Timestamp` now support an ``ambiguous`` argument for handling datetimes that are rounded to ambiguous times (:issue:`18946`)
@@ -910,6 +909,8 @@ Other API Changes
   has an improved ``KeyError`` message, and will not fail on duplicate column names with ``drop=True``. (:issue:`22484`)
 - Slicing a single row of a DataFrame with multiple ExtensionArrays of the same type now preserves the dtype, rather than coercing to object (:issue:`22784`)
 - :class:`DateOffset` attribute `_cacheable` and method `_should_cache` have been removed (:issue:`23118`)
+- Comparing :class:`Timedelta` to be less or greater than unknown types now raises a ``TypeError`` instead of returning ``False`` (:issue:`20829`)
+- :meth:`Index.hasnans` and :meth:`Series.hasnans` now always return a python boolean. Previously, a python or a numpy boolean could be returned, depending on circumstances (:issue:`23294`).
 
 .. _whatsnew_0240.deprecations:
 
@@ -941,13 +942,22 @@ Removal of prior version deprecations/changes
 - Removal of the previously deprecated module ``pandas.core.datetools`` (:issue:`14105`, :issue:`14094`)
 - Strings passed into :meth:`DataFrame.groupby` that refer to both column and index levels will raise a ``ValueError`` (:issue:`14432`)
 - :meth:`Index.repeat` and :meth:`MultiIndex.repeat` have renamed the ``n`` argument to ``repeats`` (:issue:`14645`)
+- The ``Series`` constructor and ``.astype`` method will now raise a ``ValueError`` if timestamp dtypes are passed in without a frequency (e.g. ``np.datetime64``) for the ``dtype`` parameter (:issue:`15987`)
 - Removal of the previously deprecated ``as_indexer`` keyword completely from ``str.match()`` (:issue:`22356`, :issue:`6581`)
+- The modules ``pandas.types``, ``pandas.computation``, and ``pandas.util.decorators`` have been removed (:issue:`16157`, :issue:`16250`)
 - Removed the ``pandas.formats.style`` shim for :class:`pandas.io.formats.style.Styler` (:issue:`16059`)
+- :func:`pandas.pnow`, :func:`pandas.match`, :func:`pandas.groupby`, :func:`pd.get_store`, ``pd.Expr``, and ``pd.Term`` have been removed (:issue:`15538`, :issue:`15940`)
 - :meth:`Categorical.searchsorted` and :meth:`Series.searchsorted` have renamed the ``v`` argument to ``value`` (:issue:`14645`)
+- ``pandas.parser``, ``pandas.lib``, and ``pandas.tslib`` have been removed (:issue:`15537`)
 - :meth:`TimedeltaIndex.searchsorted`, :meth:`DatetimeIndex.searchsorted`, and :meth:`PeriodIndex.searchsorted` have renamed the ``key`` argument to ``value`` (:issue:`14645`)
+- :meth:`DataFrame.consolidate` and :meth:`Series.consolidate` have been removed (:issue:`15501`)
 - Removal of the previously deprecated module ``pandas.json`` (:issue:`19944`)
+- The module ``pandas.tools`` has been removed (:issue:`15358`, :issue:`16005`)
 - :meth:`SparseArray.get_values` and :meth:`SparseArray.to_dense` have dropped the ``fill`` parameter (:issue:`14686`)
+- :meth:`DataFrame.sortlevel` and :meth:`Series.sortlevel` have been removed (:issue:`15099`)
 - :meth:`SparseSeries.to_dense` has dropped the ``sparse_only`` parameter (:issue:`14686`)
+- :meth:`DataFrame.astype` and :meth:`Series.astype` have renamed the ``raise_on_error`` argument to ``errors`` (:issue:`14967`)
+- ``is_sequence``, ``is_any_int_dtype``, and ``is_floating_dtype`` have been removed from ``pandas.api.types`` (:issue:`16163`, :issue:`16189`)
 
 .. _whatsnew_0240.performance:
 
@@ -1021,6 +1031,8 @@ Datetimelike
 - Bug in :class:`PeriodIndex` where adding or subtracting a :class:`timedelta` or :class:`Tick` object produced incorrect results (:issue:`22988`)
 - Bug in :func:`date_range` when decrementing a start date to a past end date by a negative frequency (:issue:`23270`)
 - Bug in :func:`DataFrame.combine` with datetimelike values raising a TypeError (:issue:`23079`)
+- Bug in :func:`date_range` with frequency of ``Day`` or higher where dates sufficiently far in the future could wrap around to the past instead of raising ``OutOfBoundsDatetime`` (:issue:`14187`)
+- Bug in :class:`PeriodIndex` with attribute ``freq.n`` greater than 1 where adding a :class:`DateOffset` object would return incorrect results (:issue:`23215`)
 
 Timedelta
 ^^^^^^^^^
@@ -1032,7 +1044,8 @@ Timedelta
 - Bug in :class:`TimedeltaIndex` incorrectly allowing indexing with ``Timestamp`` object (:issue:`20464`)
 - Fixed bug where subtracting :class:`Timedelta` from an object-dtyped array would raise ``TypeError`` (:issue:`21980`)
 - Fixed bug in adding a :class:`DataFrame` with all-`timedelta64[ns]` dtypes to a :class:`DataFrame` with all-integer dtypes returning incorrect results instead of raising ``TypeError`` (:issue:`22696`)
-
+- Bug in :class:`TimedeltaIndex` where adding a timezone-aware datetime scalar incorrectly returned a timezone-naive :class:`DatetimeIndex` (:issue:`23215`)
+- Bug in :class:`TimedeltaIndex` where adding ``np.timedelta64('NaT')`` incorrectly returned an all-`NaT` :class:`DatetimeIndex` instead of an all-`NaT` :class:`TimedeltaIndex` (:issue:`23215`)
 
 Timezones
 ^^^^^^^^^
@@ -1062,7 +1075,7 @@ Offsets
 
 - Bug in :class:`FY5253` where date offsets could incorrectly raise an ``AssertionError`` in arithmetic operatons (:issue:`14774`)
 - Bug in :class:`DateOffset` where keyword arguments ``week`` and ``milliseconds`` were accepted and ignored.  Passing these will now raise ``ValueError`` (:issue:`19398`)
--
+- Bug in adding :class:`DateOffset` with :class:`DataFrame` or :class:`PeriodIndex` incorrectly raising ``TypeError`` (:issue:`23215`)
 
 Numeric
 ^^^^^^^
@@ -1079,6 +1092,7 @@ Numeric
 - Bug in :meth:`DataFrame.astype` to extension dtype may raise ``AttributeError`` (:issue:`22578`)
 - Bug in :class:`DataFrame` with ``timedelta64[ns]`` dtype arithmetic operations with ``ndarray`` with integer dtype incorrectly treating the narray as ``timedelta64[ns]`` dtype (:issue:`23114`)
 - Bug in :meth:`Series.rpow` with object dtype ``NaN`` for ``1 ** NA`` instead of ``1`` (:issue:`22922`).
+- :meth:`Series.agg` can now handle numpy NaN-aware methods like :func:`numpy.nansum` (:issue:`19629`)
 
 Strings
 ^^^^^^^

pandas/__init__.py

@@ -45,37 +45,11 @@
 from pandas.core.computation.api import *
 from pandas.core.reshape.api import *
 
-# deprecate tools.plotting, plot_params and scatter_matrix on the top namespace
-import pandas.tools.plotting
-plot_params = pandas.plotting._style._Options(deprecated=True)
-# do not import deprecate to top namespace
-scatter_matrix = pandas.util._decorators.deprecate(
-    'pandas.scatter_matrix', pandas.plotting.scatter_matrix, '0.20.0',
-    'pandas.plotting.scatter_matrix')
-
 from pandas.util._print_versions import show_versions
 from pandas.io.api import *
 from pandas.util._tester import test
 import pandas.testing
 
-# extension module deprecations
-from pandas.util._depr_module import _DeprecatedModule
-
-parser = _DeprecatedModule(deprmod='pandas.parser',
-                           removals=['na_values'],
-                           moved={'CParserError': 'pandas.errors.ParserError'})
-lib = _DeprecatedModule(deprmod='pandas.lib', deprmodto=False,
-                        moved={'Timestamp': 'pandas.Timestamp',
-                               'Timedelta': 'pandas.Timedelta',
-                               'NaT': 'pandas.NaT',
-                               'infer_dtype': 'pandas.api.types.infer_dtype'})
-tslib = _DeprecatedModule(deprmod='pandas.tslib',
-                          moved={'Timestamp': 'pandas.Timestamp',
-                                 'Timedelta': 'pandas.Timedelta',
-                                 'NaT': 'pandas.NaT',
-                                 'NaTType': 'type(pandas.NaT)',
-                                 'OutOfBoundsDatetime': 'pandas.errors.OutOfBoundsDatetime'})
-
 # use the closest tagged version if possible
 from ._version import get_versions
 v = get_versions()

pandas/_libs/tslibs/offsets.pyx

@@ -345,8 +345,8 @@ class _BaseOffset(object):
         return {name: kwds[name] for name in kwds if kwds[name] is not None}
 
     def __add__(self, other):
-        if getattr(other, "_typ", None) in ["datetimeindex",
-                                            "series", "period"]:
+        if getattr(other, "_typ", None) in ["datetimeindex", "periodindex",
+                                            "series", "period", "dataframe"]:
             # defer to the other class's implementation
             return other + self
         try:

pandas/_libs/tslibs/timedeltas.pyx

@@ -1218,6 +1218,12 @@ class Timedelta(_Timedelta):
                 return other.delta * self
             return NotImplemented
 
+        elif util.is_nan(other):
+            # i.e. np.nan, but also catch np.float64("NaN") which would
+            #  otherwise get caught by the hasattr(other, "dtype") branch
+            #  incorrectly return a np.timedelta64 object.
+            return NaT
+
         elif hasattr(other, 'dtype'):
             # ndarray-like
             return other * self.to_timedelta64()
@@ -1245,6 +1251,12 @@ class Timedelta(_Timedelta):
             # convert to Timedelta below
             pass
 
+        elif util.is_nan(other):
+            # i.e. np.nan, but also catch np.float64("NaN") which would
+            #  otherwise get caught by the hasattr(other, "dtype") branch
+            #  incorrectly return a np.timedelta64 object.
+            return NaT
+
         elif hasattr(other, 'dtype'):
             return self.to_timedelta64() / other
 

pandas/computation/__init__.py

@@ -14,7 +14,7 @@
                                MultiIndex, IntervalIndex,
                                TimedeltaIndex, DatetimeIndex,
                                PeriodIndex, NaT)
-from pandas.core.indexes.period import Period, period_range, pnow
+from pandas.core.indexes.period import Period, period_range
 from pandas.core.indexes.timedeltas import Timedelta, timedelta_range
 from pandas.core.indexes.datetimes import Timestamp, date_range, bdate_range
 from pandas.core.indexes.interval import Interval, interval_range
@@ -36,27 +36,6 @@
                                 describe_option, option_context, options)
 
 
-# deprecation, xref #13790
-def match(*args, **kwargs):
-
-    import warnings
-    warnings.warn("pd.match() is deprecated and will be removed "
-                  "in a future version",
-                  FutureWarning, stacklevel=2)
-    from pandas.core.algorithms import match
-    return match(*args, **kwargs)
-
-
-def groupby(*args, **kwargs):
-    import warnings
-
-    warnings.warn("pd.groupby() is deprecated and will be removed; "
-                  "Please use the Series.groupby() or "
-                  "DataFrame.groupby() methods",
-                  FutureWarning, stacklevel=2)
-    return args[0].groupby(*args[1:], **kwargs)
-
-
 # Deprecation: xref gh-16747
 class TimeGrouper(object):