Skip to content

Series.values deprecation of converts and drops #58856

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v3.0.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,7 @@ starting with 3.0, so it can be safely removed from your code.
Other Deprecations
^^^^^^^^^^^^^^^^^^

- Deprecated :meth:`series.values`, now don't drops the ``tz`` from ``dt64tz``, converts ``interval`` to ``object``, converts ``period`` to ``object``.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be Series (not series) throughout this PR.

- Deprecated :meth:`Timestamp.utcfromtimestamp`, use ``Timestamp.fromtimestamp(ts, "UTC")`` instead (:issue:`56680`)
- Deprecated :meth:`Timestamp.utcnow`, use ``Timestamp.now("UTC")`` instead (:issue:`56680`)
- Deprecated allowing non-keyword arguments in :meth:`DataFrame.all`, :meth:`DataFrame.min`, :meth:`DataFrame.max`, :meth:`DataFrame.sum`, :meth:`DataFrame.prod`, :meth:`DataFrame.mean`, :meth:`DataFrame.median`, :meth:`DataFrame.sem`, :meth:`DataFrame.var`, :meth:`DataFrame.std`, :meth:`DataFrame.skew`, :meth:`DataFrame.kurt`, :meth:`Series.all`, :meth:`Series.min`, :meth:`Series.max`, :meth:`Series.sum`, :meth:`Series.prod`, :meth:`Series.mean`, :meth:`Series.median`, :meth:`Series.sem`, :meth:`Series.var`, :meth:`Series.std`, :meth:`Series.skew`, and :meth:`Series.kurt`. (:issue:`57087`)
Expand Down
10 changes: 10 additions & 0 deletions pandas/core/internals/blocks.py
Original file line number Diff line number Diff line change
Expand Up @@ -2349,11 +2349,21 @@ def external_values(values: ArrayLike) -> ArrayLike:
proper extension array).
"""
if isinstance(values, (PeriodArray, IntervalArray)):
warnings.warn(
"series.values will stop converting tz from dt64tz, interval to object and period to object",
FutureWarning,
stacklevel=find_stack_level(),
)
Comment on lines +2352 to +2356
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you make this warning specific to period / interval. In other words, if they have period data, they should just see the message for period.

return values.astype(object)
elif isinstance(values, (DatetimeArray, TimedeltaArray)):
# NB: for datetime64tz this is different from np.asarray(values), since
# that returns an object-dtype ndarray of Timestamps.
# Avoid raising in .astype in casting from dt64tz to dt64
warnings.warn(
"series.values will stop converting tz from dt64tz, interval to object and period to object",
FutureWarning,
stacklevel=find_stack_level(),
)
Comment on lines +2362 to +2366
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the data is tz-agnostic, then there will be no behavior change. So users should not see a warning.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How can I see if the data is tz-agnostic?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe isinstance(values.dtype, pd.DatetimeTZDtype)

values = values._ndarray

if isinstance(values, np.ndarray):
Expand Down
12 changes: 10 additions & 2 deletions pandas/tests/arithmetic/test_datetime64.py
Original file line number Diff line number Diff line change
Expand Up @@ -2329,9 +2329,17 @@ def test_dti_add_series(self, tz_naive_fixture, names):
tm.assert_series_equal(result2, expected)

expected = index + Timedelta(seconds=5)
result3 = ser.values + index
with tm.assert_produces_warning(
FutureWarning,
match="series.values will stop converting tz from dt64tz, interval to object and period to object",
):
result3 = ser.values + index
tm.assert_index_equal(result3, expected)
result4 = index + ser.values
with tm.assert_produces_warning(
FutureWarning,
match="series.values will stop converting tz from dt64tz, interval to object and period to object",
):
result4 = index + ser.values
tm.assert_index_equal(result4, expected)

@pytest.mark.parametrize("op", [operator.add, roperator.radd, operator.sub])
Expand Down
6 changes: 5 additions & 1 deletion pandas/tests/arrays/test_timedeltas.py
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,11 @@ def test_total_seconds_nanoseconds(self):
# issue #48521
start_time = pd.Series(["2145-11-02 06:00:00"]).astype("datetime64[ns]")
end_time = pd.Series(["2145-11-02 07:06:00"]).astype("datetime64[ns]")
expected = (end_time - start_time).values / np.timedelta64(1, "s")
with tm.assert_produces_warning(
FutureWarning,
match="series.values will stop converting tz from dt64tz, interval to object and period to object",
):
expected = (end_time - start_time).values / np.timedelta64(1, "s")
result = (end_time - start_time).dt.total_seconds().values
assert result == expected

Expand Down
6 changes: 5 additions & 1 deletion pandas/tests/base/test_value_counts.py
Original file line number Diff line number Diff line change
Expand Up @@ -254,7 +254,11 @@ def test_value_counts_datetime64(index_or_series, unit):

# with NaT
s = df["dt"].copy()
s = klass(list(s.values) + [pd.NaT] * 4)
with tm.assert_produces_warning(
FutureWarning,
match="series.values will stop converting tz from dt64tz, interval to object and period to object",
):
s = klass(list(s.values) + [pd.NaT] * 4)
if klass is Series:
s = s.dt.as_unit(unit)
else:
Expand Down
16 changes: 10 additions & 6 deletions pandas/tests/computation/test_eval.py
Original file line number Diff line number Diff line change
Expand Up @@ -1507,12 +1507,16 @@ def test_nested_period_index_subscript_expression(self):
def test_date_boolean(self, engine, parser):
df = DataFrame(np.random.default_rng(2).standard_normal((5, 3)))
df["dates1"] = date_range("1/1/2012", periods=5)
res = self.eval(
"df.dates1 < 20130101",
local_dict={"df": df},
engine=engine,
parser=parser,
)
with tm.assert_produces_warning(
FutureWarning,
match="series.values will stop converting tz from dt64tz, interval to object and period to object",
):
res = self.eval(
"df.dates1 < 20130101",
local_dict={"df": df},
engine=engine,
parser=parser,
)
Comment on lines +1514 to +1519
Copy link
Member

@rhshadrach rhshadrach Jun 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "user" is not using .values here, they should not see a warning (at least in regards to using .values). Will the behavior here actually change for a user?

expec = df.dates1 < "20130101"
tm.assert_series_equal(res, expec)

Expand Down
6 changes: 5 additions & 1 deletion pandas/tests/copy_view/test_astype.py
Original file line number Diff line number Diff line change
Expand Up @@ -187,7 +187,11 @@ def test_astype_arrow_timestamp():
},
dtype="M8[ns]",
)
result = df.astype("timestamp[ns][pyarrow]")
with tm.assert_produces_warning(
FutureWarning,
match="series.values will stop converting tz from dt64tz, interval to object and period to object",
):
result = df.astype("timestamp[ns][pyarrow]")
assert not result._mgr._has_no_reference(0)
if pa_version_under12p0:
assert not np.shares_memory(
Expand Down
12 changes: 10 additions & 2 deletions pandas/tests/frame/indexing/test_setitem.py
Original file line number Diff line number Diff line change
Expand Up @@ -305,7 +305,11 @@ def test_frame_setitem_datetime64_col_other_units(self, unit):
df[unit] = vals

assert df[unit].dtype == ex_vals.dtype
assert (df[unit].values == ex_vals).all()
with tm.assert_produces_warning(
FutureWarning,
match="series.values will stop converting tz from dt64tz, interval to object and period to object",
):
assert (df[unit].values == ex_vals).all()

@pytest.mark.parametrize("unit", ["h", "m", "s", "ms", "D", "M", "Y"])
def test_frame_setitem_existing_datetime64_col_other_units(self, unit):
Expand All @@ -322,7 +326,11 @@ def test_frame_setitem_existing_datetime64_col_other_units(self, unit):

# We overwrite existing dt64 column with new, non-nano dt64 vals
df["dates"] = vals
assert (df["dates"].values == ex_vals).all()
with tm.assert_produces_warning(
FutureWarning,
match="series.values will stop converting tz from dt64tz, interval to object and period to object",
):
assert (df["dates"].values == ex_vals).all()

def test_setitem_dt64tz(self, timezone_frame):
df = timezone_frame
Expand Down
12 changes: 10 additions & 2 deletions pandas/tests/frame/methods/test_convert_dtypes.py
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,11 @@ def test_pyarrow_dtype_backend(self):
"g": pd.Series(pd.timedelta_range("1D", periods=3)),
}
)
result = df.convert_dtypes(dtype_backend="pyarrow")
with tm.assert_produces_warning(
FutureWarning,
match="series.values will stop converting tz from dt64tz, interval to object and period to object",
):
result = df.convert_dtypes(dtype_backend="pyarrow")
expected = pd.DataFrame(
{
"a": pd.arrays.ArrowExtensionArray(
Expand Down Expand Up @@ -176,7 +180,11 @@ def test_convert_dtypes_pyarrow_timestamp(self):
# GH 54191
pytest.importorskip("pyarrow")
ser = pd.Series(pd.date_range("2020-01-01", "2020-01-02", freq="1min"))
expected = ser.astype("timestamp[ms][pyarrow]")
with tm.assert_produces_warning(
FutureWarning,
match="series.values will stop converting tz from dt64tz, interval to object and period to object",
):
expected = ser.astype("timestamp[ms][pyarrow]")
result = expected.convert_dtypes(dtype_backend="pyarrow")
tm.assert_series_equal(result, expected)

Expand Down
6 changes: 5 additions & 1 deletion pandas/tests/frame/methods/test_duplicated.py
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,11 @@ def test_frame_datetime64_duplicated():
dates = date_range("2010-07-01", end="2010-08-05")

tst = DataFrame({"symbol": "AAA", "date": dates})
result = tst.duplicated(["date", "symbol"])
with tm.assert_produces_warning(
FutureWarning,
match="series.values will stop converting tz from dt64tz, interval to object and period to object",
):
result = tst.duplicated(["date", "symbol"])
assert (-result).all()

tst = DataFrame({"date": dates})
Expand Down
12 changes: 10 additions & 2 deletions pandas/tests/frame/test_block_internals.py
Original file line number Diff line number Diff line change
Expand Up @@ -293,13 +293,21 @@ def test_consolidate_datetime64(self):
)

ser_starting = df.starting
ser_starting.index = ser_starting.values
with tm.assert_produces_warning(
FutureWarning,
match="series.values will stop converting tz from dt64tz, interval to object and period to object",
):
ser_starting.index = ser_starting.values
ser_starting = ser_starting.tz_localize("US/Eastern")
ser_starting = ser_starting.tz_convert("UTC")
ser_starting.index.name = "starting"

ser_ending = df.ending
ser_ending.index = ser_ending.values
with tm.assert_produces_warning(
FutureWarning,
match="series.values will stop converting tz from dt64tz, interval to object and period to object",
):
ser_ending.index = ser_ending.values
ser_ending = ser_ending.tz_localize("US/Eastern")
ser_ending = ser_ending.tz_convert("UTC")
ser_ending.index.name = "ending"
Expand Down
Loading
Loading