Skip to content

BUG: parsing mixed-offset Timestamps with errors='ignore' and no format raises #50585

Closed
@MarcoGorelli

Description

@MarcoGorelli

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

In [1]: to_datetime([Timestamp('2020').tz_localize('Europe/London'), Timestamp('2020').tz_localize('US/Pacific')], errors='ignore')
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[1], line 1
----> 1 to_datetime([Timestamp('2020').tz_localize('Europe/London'), Timestamp('2020').tz_localize('US/Pacific')], errors='ignore')

File ~/pandas-dev/pandas/core/tools/datetimes.py:1063, in to_datetime(arg, errors, dayfirst, yearfirst, utc, format, exact, unit, infer_datetime_format, origin, cache)
   1061         result = _convert_and_box_cache(argc, cache_array)
   1062     else:
-> 1063         result = convert_listlike(argc, format)
   1064 else:
   1065     result = convert_listlike(np.array([arg]), format)[0]

File ~/pandas-dev/pandas/core/tools/datetimes.py:437, in _convert_listlike_datetimes(arg, format, name, utc, unit, errors, dayfirst, yearfirst, exact)
    434 if format is not None:
    435     return _array_strptime_with_fallback(arg, name, utc, format, exact, errors)
--> 437 result, tz_parsed = objects_to_datetime64ns(
    438     arg,
    439     dayfirst=dayfirst,
    440     yearfirst=yearfirst,
    441     utc=utc,
    442     errors=errors,
    443     allow_object=True,
    444 )
    446 if tz_parsed is not None:
    447     # We can take a shortcut since the datetime64 numpy array
    448     # is in UTC
    449     dta = DatetimeArray(result, dtype=tz_to_dtype(tz_parsed))

File ~/pandas-dev/pandas/core/arrays/datetimes.py:2158, in objects_to_datetime64ns(data, dayfirst, yearfirst, utc, errors, allow_object)
   2156 order: Literal["F", "C"] = "F" if flags.f_contiguous else "C"
   2157 try:
-> 2158     result, tz_parsed = tslib.array_to_datetime(
   2159         data.ravel("K"),
   2160         errors=errors,
   2161         utc=utc,
   2162         dayfirst=dayfirst,
   2163         yearfirst=yearfirst,
   2164     )
   2165     result = result.reshape(data.shape, order=order)
   2166 except OverflowError as err:
   2167     # Exception is raised when a part of date is greater than 32 bit signed int

File ~/pandas-dev/pandas/_libs/tslib.pyx:441, in pandas._libs.tslib.array_to_datetime()
    439 @cython.wraparound(False)
    440 @cython.boundscheck(False)
--> 441 cpdef array_to_datetime(
    442     ndarray[object] values,
    443     str errors="raise",

File ~/pandas-dev/pandas/_libs/tslib.pyx:521, in pandas._libs.tslib.array_to_datetime()
    519 else:
    520     found_naive = True
--> 521 tz_out = convert_timezone(
    522     val.tzinfo,
    523     tz_out,

File ~/pandas-dev/pandas/_libs/tslibs/conversion.pyx:725, in pandas._libs.tslibs.conversion.convert_timezone()
    723                      "datetime64 unless utc=True")
    724 elif tz_out is not None and not tz_compare(tz_out, tz_in):
--> 725     raise ValueError("Tz-aware datetime.datetime "
    726                      "cannot be converted to "
    727                      "datetime64 unless utc=True")

ValueError: Tz-aware datetime.datetime cannot be converted to datetime64 unless utc=True

In [2]: to_datetime([Timestamp('2020').tz_localize('Europe/London'), Timestamp('2020').tz_localize('US/Pacific')], errors='ignore', format='%Y-%m
   ...: -%d')
Out[2]: Index([2020-01-01 00:00:00+00:00, 2020-01-01 00:00:00-08:00], dtype='object')

Issue Description

It shouldn't raise with errors='ignore'

Expected Behavior

Index([2020-01-01 00:00:00+00:00, 2020-01-01 00:00:00-08:00], dtype='object')

Installed Versions

INSTALLED VERSIONS

commit : 8acd314
python : 3.8.16.final.0
python-bits : 64
OS : Linux
OS-release : 5.10.102.1-microsoft-standard-WSL2
Version : #1 SMP Wed Mar 2 00:30:59 UTC 2022
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : en_GB.UTF-8
LOCALE : en_GB.UTF-8

pandas : 2.0.0.dev0+1049.g8acd31495a
numpy : 1.23.5
pytz : 2022.7
dateutil : 2.8.2
setuptools : 65.6.3
pip : 22.3.1
Cython : 0.29.32
pytest : 7.2.0
hypothesis : 6.61.0
sphinx : 5.3.0
blosc : 1.11.1
feather : None
xlsxwriter : 3.0.6
lxml.etree : 4.9.2
html5lib : 1.1
pymysql : 1.0.2
psycopg2 : 2.9.5
jinja2 : 3.1.2
IPython : 8.8.0
pandas_datareader: None
bs4 : 4.11.1
bottleneck : 1.3.5
brotli :
fastparquet : 2022.12.0
fsspec : 2022.11.0
gcsfs : 2022.11.0
matplotlib : 3.6.2
numba : 0.56.4
numexpr : 2.8.4
odfpy : None
openpyxl : 3.0.10
pandas_gbq : None
pyarrow : 10.0.1
pyreadstat : 1.2.0
pyxlsb : 1.0.10
s3fs : 2022.11.0
scipy : 1.10.0
snappy :
sqlalchemy : 1.4.46
tables : 3.8.0
tabulate : 0.9.0
xarray : 2022.12.0
xlrd : 2.0.1
zstandard : 0.19.0
tzdata : 2022.7
qtpy : None
pyqt5 : None
None

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions