Skip to content

CLN: Assorted cleanups #30260

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 23 commits into from
Dec 15, 2019
Merged
Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
24e2ddd
CLN: pytables annotations and docstrings
jbrockmendel Dec 2, 2019
b16b4cc
Merge branch 'master' of https://github.com/pandas-dev/pandas into cl…
jbrockmendel Dec 2, 2019
ec7ddc5
sig cleanup
jbrockmendel Dec 2, 2019
f6d32d2
Merge branch 'master' of https://github.com/pandas-dev/pandas into cl…
jbrockmendel Dec 3, 2019
c4d90a2
Merge branch 'master' of https://github.com/pandas-dev/pandas into cl…
jbrockmendel Dec 3, 2019
3002f1f
Merge branch 'master' of https://github.com/pandas-dev/pandas into cl…
jbrockmendel Dec 3, 2019
069e966
Merge branch 'master' of https://github.com/pandas-dev/pandas into cl…
jbrockmendel Dec 4, 2019
ec6125a
Merge branch 'master' of https://github.com/pandas-dev/pandas into cl…
jbrockmendel Dec 5, 2019
a9b52ae
Merge branch 'master' of https://github.com/pandas-dev/pandas into cl…
jbrockmendel Dec 5, 2019
45b7ac3
Merge branch 'master' of https://github.com/pandas-dev/pandas into cl…
jbrockmendel Dec 5, 2019
9801a75
Merge branch 'master' of https://github.com/pandas-dev/pandas into cl…
jbrockmendel Dec 6, 2019
0a64173
Merge branch 'master' of https://github.com/pandas-dev/pandas into cl…
jbrockmendel Dec 6, 2019
1e11396
Merge branch 'master' of https://github.com/pandas-dev/pandas into cl…
jbrockmendel Dec 6, 2019
e40f054
Merge branch 'master' of https://github.com/pandas-dev/pandas into cl…
jbrockmendel Dec 6, 2019
009c893
Merge branch 'master' of https://github.com/pandas-dev/pandas into cl…
jbrockmendel Dec 12, 2019
17fa54b
Merge branch 'master' of https://github.com/pandas-dev/pandas into cl…
jbrockmendel Dec 12, 2019
74f31f2
CLN: assorted cleanups
jbrockmendel Dec 12, 2019
6ca4461
Merge branch 'master' of https://github.com/pandas-dev/pandas into cl…
jbrockmendel Dec 12, 2019
80d32e1
requested paramertization from #30222
jbrockmendel Dec 12, 2019
41a3f1a
docstrings
jbrockmendel Dec 13, 2019
554cb0d
Merge branch 'master' of https://github.com/pandas-dev/pandas into cl…
jbrockmendel Dec 13, 2019
560a6ef
Merge branch 'master' of https://github.com/pandas-dev/pandas into cl…
jbrockmendel Dec 13, 2019
c6cd691
typo fixup
jbrockmendel Dec 13, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 24 additions & 8 deletions pandas/core/dtypes/cast.py
Original file line number Diff line number Diff line change
Expand Up @@ -244,13 +244,13 @@ def maybe_upcast_putmask(result: np.ndarray, mask: np.ndarray, other):
necessary.
mask : boolean ndarray
other : scalar
The source value
The source value.

Returns
-------
result : ndarray
changed : boolean
Set to true if the result array was upcasted
changed : bool
Set to true if the result array was upcasted.

Examples
--------
Expand Down Expand Up @@ -337,6 +337,21 @@ def changeit():


def maybe_promote(dtype, fill_value=np.nan):
"""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should type at some point

Find the minimal dtype that can hold both the given dtype and fill_value.

Parameters
----------
dtype : np.dtype or ExceptionDtype
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ExtensionDtype?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep, will change

fill_value : scalar, default np.nan

Returns
-------
dtype
Upcasted from dtype argument if necessary.
fill_value
Upcasted from fill_value argument if necessary.
"""
if not is_scalar(fill_value) and not is_object_dtype(dtype):
# with object dtype there is nothing to promote, and the user can
# pass pretty much any weird fill_value they like
Expand Down Expand Up @@ -592,11 +607,11 @@ def infer_dtype_from_scalar(val, pandas_dtype: bool = False):

def infer_dtype_from_array(arr, pandas_dtype: bool = False):
"""
Infer the dtype from a scalar or array.
Infer the dtype from an array.

Parameters
----------
arr : scalar or array
arr : array
pandas_dtype : bool, default False
whether to infer dtype including pandas extension types.
If False, array belongs to pandas extension types
Expand All @@ -622,7 +637,6 @@ def infer_dtype_from_array(arr, pandas_dtype: bool = False):

>>> infer_dtype_from_array([1, '1'])
(numpy.object_, [1, '1'])

"""

if isinstance(arr, np.ndarray):
Expand Down Expand Up @@ -686,10 +700,12 @@ def maybe_upcast(values, fill_value=np.nan, dtype=None, copy: bool = False):

Parameters
----------
values : the ndarray that we want to maybe upcast
values : ndarray or ExtensionArray
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ideally type these at some point

The array that we want to maybe upcast.
fill_value : what we want to fill with
dtype : if None, then use the dtype of the values, else coerce to this type
copy : if True always make a copy even if no upcast is required
copy : bool, default True
If True always make a copy even if no upcast is required.
"""
if not is_scalar(fill_value) and not is_object_dtype(values.dtype):
# We allow arbitrary fill values for object dtype
Expand Down
2 changes: 1 addition & 1 deletion pandas/core/frame.py
Original file line number Diff line number Diff line change
Expand Up @@ -4354,7 +4354,7 @@ def _maybe_casted_values(index, labels=None):
values = values._data

if mask.any():
values, changed = maybe_upcast_putmask(values, mask, np.nan)
values, _ = maybe_upcast_putmask(values, mask, np.nan)

if issubclass(values_type, DatetimeLikeArray):
values = values_type(values, dtype=values_dtype)
Expand Down
3 changes: 2 additions & 1 deletion pandas/core/indexes/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -331,11 +331,12 @@ def __new__(

# extension dtype
elif is_extension_array_dtype(data) or is_extension_array_dtype(dtype):
data = np.asarray(data)
if not (dtype is None or is_object_dtype(dtype)):
# coerce to the provided dtype
ea_cls = dtype.construct_array_type()
data = ea_cls._from_sequence(data, dtype=dtype, copy=False)
else:
data = np.asarray(data, dtype=object)

# coerce to the object dtype
data = data.astype(object)
Expand Down
2 changes: 1 addition & 1 deletion pandas/core/nanops.py
Original file line number Diff line number Diff line change
Expand Up @@ -314,7 +314,7 @@ def _get_values(

# promote if needed
else:
values, changed = maybe_upcast_putmask(values, mask, fill_value)
values, _ = maybe_upcast_putmask(values, mask, fill_value)

# return a platform independent precision dtype
dtype_max = dtype
Expand Down
2 changes: 1 addition & 1 deletion pandas/core/ops/array_ops.py
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ def masked_arith_op(x, y, op):
with np.errstate(all="ignore"):
result[mask] = op(xrav[mask], y)

result, changed = maybe_upcast_putmask(result, ~mask, np.nan)
result, _ = maybe_upcast_putmask(result, ~mask, np.nan)
result = result.reshape(x.shape) # 2D compat
return result

Expand Down
6 changes: 4 additions & 2 deletions pandas/core/strings.py
Original file line number Diff line number Diff line change
Expand Up @@ -74,10 +74,12 @@ def cat_core(list_of_columns: List, sep: str):
"""
if sep == "":
# no need to interleave sep if it is empty
return np.sum(list_of_columns, axis=0)
arr_of_cols = np.asarray(list_of_columns, dtype=object)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this cleaning up?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is for when numpy re-institutes the make-object-explicit thing from #30035.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think should hold off on these changes then until that comes back

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

they reverted it specifically to give downstream time to update without breaking CIs

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great this is good

return np.sum(arr_of_cols, axis=0)
list_with_sep = [sep] * (2 * len(list_of_columns) - 1)
list_with_sep[::2] = list_of_columns
return np.sum(list_with_sep, axis=0)
arr_with_sep = np.asarray(list_with_sep)
return np.sum(arr_with_sep, axis=0)


def cat_safe(list_of_columns: List, sep: str):
Expand Down
Loading