Skip to content

TYP: expand acceptable types for pd.to_datetime() #46273

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 21 commits into from
Mar 11, 2022
Merged
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
9e57d6d
fix column_arrays for array manager
Dr-Irv Dec 21, 2021
2abbc57
merge with upstream/main
Dr-Irv Jan 15, 2022
99158b1
Merge remote-tracking branch 'upstream/main'
Dr-Irv Jan 24, 2022
0f4130d
Merge remote-tracking branch 'upstream/main'
Dr-Irv Feb 4, 2022
24ecfe2
Merge remote-tracking branch 'upstream/main'
Dr-Irv Feb 8, 2022
64f00d5
Merge remote-tracking branch 'upstream/main'
Dr-Irv Feb 9, 2022
1ee6e00
Merge remote-tracking branch 'upstream/main'
Dr-Irv Feb 19, 2022
5761bf2
Merge remote-tracking branch 'upstream/main'
Dr-Irv Feb 23, 2022
84b119f
remove dead code in arrays/interval.py
Dr-Irv Feb 23, 2022
5232375
Merge remote-tracking branch 'upstream/main'
Dr-Irv Feb 25, 2022
64feddd
Undo Revert "remove dead code in arrays/interval.py"
Dr-Irv Feb 25, 2022
1cc352a
Merge remote-tracking branch 'upstream/main'
Dr-Irv Feb 27, 2022
06a4028
Merge remote-tracking branch 'upstream/main'
Dr-Irv Feb 27, 2022
957372c
Merge remote-tracking branch 'upstream/main'
Dr-Irv Mar 8, 2022
8801db6
add acceptable types to pd.to_datetime
Dr-Irv Mar 8, 2022
ff7586f
to_datetime on scalar returns Timestamp in overload
Dr-Irv Mar 8, 2022
54d937f
Merge remote-tracking branch 'upstream/main' into typetodatetime
Dr-Irv Mar 9, 2022
dcbde37
Merge remote-tracking branch 'upstream/main' into typetodatetime
Dr-Irv Mar 9, 2022
b451ca8
add specifics for TypedDict arg
Dr-Irv Mar 9, 2022
f119579
Merge remote-tracking branch 'upstream/main' into typetodatetime
Dr-Irv Mar 11, 2022
52836e6
use ArrayLike
Dr-Irv Mar 11, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 14 additions & 9 deletions pandas/core/tools/datetimes.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
from typing import (
TYPE_CHECKING,
Callable,
Dict,
Hashable,
List,
Tuple,
Expand Down Expand Up @@ -79,7 +80,10 @@
if TYPE_CHECKING:
from pandas._libs.tslibs.nattype import NaTType

from pandas import Series
from pandas import (
DataFrame,
Series,
)

# ---------------------------------------------------------------------
# types used in annotations
Expand All @@ -89,6 +93,7 @@
DatetimeScalar = Union[Scalar, datetime]

DatetimeScalarOrArrayConvertible = Union[DatetimeScalar, ArrayConvertible]
DictConvertible = Union[Dict[str, List], "DataFrame"]
start_caching_at = 50


Expand Down Expand Up @@ -640,13 +645,13 @@ def to_datetime(
infer_datetime_format: bool = ...,
origin=...,
cache: bool = ...,
) -> Timestamp | NaTType:
) -> Timestamp:
...


@overload
def to_datetime(
arg: Series,
arg: Series | DictConvertible,
errors: str = ...,
dayfirst: bool = ...,
yearfirst: bool = ...,
Expand All @@ -663,7 +668,7 @@ def to_datetime(

@overload
def to_datetime(
arg: list | tuple | np.ndarray,
arg: list | tuple | np.ndarray | Index | ExtensionArray,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sequence?

Copy link
Contributor Author

@Dr-Irv Dr-Irv Mar 9, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, can't use Sequence, because a string is a sequence, and that has to match the scalar type above this.

Keeping unchanged.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could replace np.ndarray | ExtensionArray with ArrayLike.

about Sequence and str: Does to_datetime require certain list/tuple elements? I'm sure that str isn't part of Sequence[datetime.time]

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to_datetime can take mixed lists/tuples of different kinds of scalars (datetimes, ints, floats, strings at a minimum), and I'm a bit hesitant to figure them all out to avoid making this too narrow. So I'm leaving this as list | tuple for now.

errors: str = ...,
dayfirst: bool = ...,
yearfirst: bool = ...,
Expand All @@ -679,7 +684,7 @@ def to_datetime(


def to_datetime(
arg: DatetimeScalarOrArrayConvertible,
arg: DatetimeScalarOrArrayConvertible | DictConvertible,
errors: str = "raise",
dayfirst: bool = False,
yearfirst: bool = False,
Expand Down Expand Up @@ -1067,10 +1072,10 @@ def to_datetime(
# "Union[float, str, datetime, List[Any], Tuple[Any, ...], ExtensionArray,
# ndarray[Any, Any], Series]"; expected "Union[List[Any], Tuple[Any, ...],
# Union[Union[ExtensionArray, ndarray[Any, Any]], Index, Series], Series]"
arg = cast(
argc = cast(
Union[list, tuple, ExtensionArray, np.ndarray, "Series", Index], arg
)
cache_array = _maybe_cache(arg, format, cache, convert_listlike)
cache_array = _maybe_cache(argc, format, cache, convert_listlike)
except OutOfBoundsDatetime:
# caching attempts to create a DatetimeIndex, which may raise
# an OOB. If that's the desired behavior, then just reraise...
Expand All @@ -1081,9 +1086,9 @@ def to_datetime(

cache_array = Series([], dtype=object) # just an empty array
if not cache_array.empty:
result = _convert_and_box_cache(arg, cache_array)
result = _convert_and_box_cache(argc, cache_array)
else:
result = convert_listlike(arg, format)
result = convert_listlike(argc, format)
else:
result = convert_listlike(np.array([arg]), format)[0]
if isinstance(arg, bool) and isinstance(result, np.bool_):
Expand Down
4 changes: 1 addition & 3 deletions pandas/io/excel/_odfreader.py
Original file line number Diff line number Diff line change
Expand Up @@ -214,9 +214,7 @@ def _get_cell_value(self, cell, convert_float: bool) -> Scalar | NaTType:
cell_value = cell.attributes.get((OFFICENS, "date-value"))
return pd.to_datetime(cell_value)
elif cell_type == "time":
# cast needed because `pd.to_datetime can return NaTType,
# but we know this is a valid time
stamp = cast(pd.Timestamp, pd.to_datetime(str(cell)))
stamp = pd.to_datetime(str(cell))
# cast needed here because Scalar doesn't include datetime.time
return cast(Scalar, stamp.time())
else:
Expand Down