CI: Numpy 1.22 caused CI to fail

[phofl]

The test

FAILED pandas/tests/frame/test_ufunc.py::test_alignment_deprecation_many_inputs

is failing.

Also we have some typing problems

Performing static analysis using mypy
pandas/core/internals/ops.py:130: error: Unused "type: ignore" comment
pandas/core/dtypes/common.py:539: error: Unused "type: ignore" comment
pandas/core/dtypes/cast.py:1127: error: Unused "type: ignore" comment
pandas/core/dtypes/cast.py:1149: error: Unused "type: ignore" comment
pandas/core/missing.py:337: error: Argument 1 to "apply_along_axis" has incompatible type "Callable[[ndarray[Any, Any]], None]"; expected "Callable[..., Union[_SupportsArray[dtype[<nothing>]], Sequence[_SupportsArray[dtype[<nothing>]]], Sequence[Sequence[_SupportsArray[dtype[<nothing>]]]], Sequence[Sequence[Sequence[_SupportsArray[dtype[<nothing>]]]]], Sequence[Sequence[Sequence[Sequence[_SupportsArray[dtype[<nothing>]]]]]]]]"  [arg-type]
pandas/core/missing.py:776: error: Argument 1 to "apply_along_axis" has incompatible type "partial[None]"; expected "Callable[..., Union[_SupportsArray[dtype[<nothing>]], Sequence[_SupportsArray[dtype[<nothing>]]], Sequence[Sequence[_SupportsArray[dtype[<nothing>]]]], Sequence[Sequence[Sequence[_SupportsArray[dtype[<nothing>]]]]], Sequence[Sequence[Sequence[Sequence[_SupportsArray[dtype[<nothing>]]]]]]]]"  [arg-type]
pandas/core/missing.py:776: note: "partial[None].__call__" has type "Callable[[VarArg(Any), KwArg(Any)], None]"
pandas/core/missing.py:782: error: Argument 2 to "apply_along_axis" has incompatible type "Union[str, int]"; expected "SupportsIndex"  [arg-type]
pandas/core/arraylike.py:271: error: Unused "type: ignore" comment
pandas/core/algorithms.py:772: error: Argument 2 to "append" has incompatible type "List[Union[str, float, Period, Timestamp, Timedelta, Any]]"; expected "Union[_SupportsArray[dtype[Any]], _NestedSequence[_SupportsArray[dtype[Any]]], bool, int, float, complex, str, bytes, _NestedSequence[Union[bool, int, float, complex, str, bytes]]]"  [arg-type]
pandas/core/algorithms.py:1071: error: Argument 1 to "__call__" of "_UFunc_Nin1_Nout1" has incompatible type "Optional[ndarray[Any, dtype[bool_]]]"; expected "Union[_SupportsArray[dtype[Any]], _NestedSequence[_SupportsArray[dtype[Any]]], bool, int, float, complex, str, bytes, _NestedSequence[Union[bool, int, float, complex, str, bytes]]]"  [arg-type]
pandas/core/strings/accessor.py:585: error: Incompatible types in assignment (expression has type "ndarray[Any, dtype[Any]]", variable has type "Series")  [assignment]
pandas/core/indexes/base.py:611: error: Unused "type: ignore" comment
pandas/core/indexes/base.py:4527: error: Argument 3 to "putmask" has incompatible type "Union[ExtensionArray, ndarray[Any, Any]]"; expected "Union[_SupportsArray[dtype[Any]], _NestedSequence[_SupportsArray[dtype[Any]]], bool, int, float, complex, str, bytes, _NestedSequence[Union[bool, int, float, complex, str, bytes]]]"  [arg-type]
pandas/core/indexes/base.py:5029: error: Item "ExtensionArray" of "Union[ExtensionArray, ndarray[Any, Any]]" has no attribute "_ndarray"  [union-attr]
pandas/core/indexes/base.py:5029: error: Item "ndarray[Any, Any]" of "Union[ExtensionArray, ndarray[Any, Any]]" has no attribute "_ndarray"  [union-attr]
pandas/core/indexes/base.py:6520: error: Incompatible types in assignment (expression has type "ExtensionArray", variable has type "ndarray[Any, dtype[Any]]")  [assignment]
pandas/core/indexes/base.py:6571: error: No overload variant of "insert" matches argument types "ndarray[Any, Any]", "int", "None"  [call-overload]
pandas/core/indexes/base.py:6571: note: Possible overload variants:
pandas/core/indexes/base.py:6571: note:     def [_SCT <: generic] insert(arr: Union[_SupportsArray[dtype[_SCT]], Sequence[_SupportsArray[dtype[_SCT]]], Sequence[Sequence[_SupportsArray[dtype[_SCT]]]], Sequence[Sequence[Sequence[_SupportsArray[dtype[_SCT]]]]], Sequence[Sequence[Sequence[Sequence[_SupportsArray[dtype[_SCT]]]]]]], obj: Union[slice, Union[_SupportsArray[dtype[Union[bool_, integer[Any]]]], _NestedSequence[_SupportsArray[dtype[Union[bool_, integer[Any]]]]], bool, int, _NestedSequence[Union[bool, int]]]], values: Union[_SupportsArray[dtype[Any]], _NestedSequence[_SupportsArray[dtype[Any]]], bool, int, float, complex, str, bytes, _NestedSequence[Union[bool, int, float, complex, str, bytes]]], axis: Optional[SupportsIndex] = ...) -> ndarray[Any, dtype[_SCT]]
pandas/core/indexes/base.py:6571: note:     def insert(arr: Union[_SupportsArray[dtype[Any]], _NestedSequence[_SupportsArray[dtype[Any]]], bool, int, float, complex, str, bytes, _NestedSequence[Union[bool, int, float, complex, str, bytes]]], obj: Union[slice, Union[_SupportsArray[dtype[Union[bool_, integer[Any]]]], _NestedSequence[_SupportsArray[dtype[Union[bool_, integer[Any]]]]], bool, int, _NestedSequence[Union[bool, int]]]], values: Union[_SupportsArray[dtype[Any]], _NestedSequence[_SupportsArray[dtype[Any]]], bool, int, float, complex, str, bytes, _NestedSequence[Union[bool, int, float, complex, str, bytes]]], axis: Optional[SupportsIndex] = ...) -> ndarray[Any, dtype[Any]]
pandas/core/arrays/masked.py:770: error: Need type annotation for "npvalues"  [var-annotated]
pandas/core/arrays/masked.py:946: error: Argument 3 to "putmask" has incompatible type "object"; expected "Union[_SupportsArray[dtype[Any]], _NestedSequence[_SupportsArray[dtype[Any]]], bool, int, float, complex, str, bytes, _NestedSequence[Union[bool, int, float, complex, str, bytes]]]"  [arg-type]
pandas/core/arrays/masked.py:1022: error: Argument 3 to "putmask" has incompatible type "object"; expected "Union[_SupportsArray[dtype[Any]], _NestedSequence[_SupportsArray[dtype[Any]]], bool, int, float, complex, str, bytes, _NestedSequence[Union[bool, int, float, complex, str, bytes]]]"  [arg-type]
pandas/core/arrays/interval.py:1642: error: Argument 1 to "in1d" has incompatible type "Union[ExtensionArray, ndarray[Any, Any], ndarray[Any, dtype[Any]]]"; expected "Union[_SupportsArray[dtype[Any]], _NestedSequence[_SupportsArray[dtype[Any]]], bool, int, float, complex, str, bytes, _NestedSequence[Union[bool, int, float, complex, str, bytes]]]"  [arg-type]
pandas/core/arrays/sparse/array.py:775: error: Need type annotation for "new_values"  [var-annotated]
pandas/core/arrays/sparse/array.py:927: error: Invalid index type "Tuple[Union[int, ellipsis], ...]" for "ndarray[Any, Any]"; expected type "Union[SupportsIndex, _SupportsArray[dtype[Union[bool_, integer[Any]]]], _NestedSequence[_SupportsArray[dtype[Union[bool_, integer[Any]]]]], _NestedSequence[Union[bool, int]], Tuple[Union[SupportsIndex, _SupportsArray[dtype[Union[bool_, integer[Any]]]], _NestedSequence[_SupportsArray[dtype[Union[bool_, integer[Any]]]]], _NestedSequence[Union[bool, int]]], ...]]"  [index]
pandas/core/arrays/sparse/array.py:1167: error: Argument 2 to "IntIndex" has incompatible type "ndarray[Any, dtype[signedinteger[_32Bit]]]"; expected "Sequence[int]"  [arg-type]
pandas/core/arrays/sparse/array.py:1345: error: Need type annotation for "sparse_values"  [var-annotated]
pandas/core/indexes/multi.py:368: error: Incompatible types in assignment (expression has type "ndarray[Any, dtype[Any]]", variable has type "List[Any]")  [assignment]
pandas/core/indexes/multi.py:1091: error: Argument 1 to "concatenate" has incompatible type "List[object]"; expected "Union[_SupportsArray[dtype[<nothing>]], Sequence[_SupportsArray[dtype[<nothing>]]], Sequence[Sequence[_SupportsArray[dtype[<nothing>]]]], Sequence[Sequence[Sequence[_SupportsArray[dtype[<nothing>]]]]], Sequence[Sequence[Sequence[Sequence[_SupportsArray[dtype[<nothing>]]]]]]]"  [arg-type]
pandas/core/indexes/multi.py:1569: error: Argument 1 to "lexsort" has incompatible type "List[Union[ExtensionArray, ndarray[Any, Any]]]"; expected "Union[_SupportsArray[dtype[Any]], _NestedSequence[_SupportsArray[dtype[Any]]], bool, int, float, complex, str, bytes, _NestedSequence[Union[bool, int, float, complex, str, bytes]]]"  [arg-type]
pandas/core/indexing.py:2018: error: Incompatible types in assignment (expression has type "ndarray[Any, dtype[signedinteger[Any]]]", variable has type "List[Any]")  [assignment]
pandas/core/indexing.py:2080: error: Incompatible types in assignment (expression has type "ndarray[Any, dtype[Any]]", variable has type "Series")  [assignment]
pandas/io/stata.py:602: error: Value of type variable "_IntType" of "iinfo" cannot be "object"  [type-var]
pandas/io/stata.py:685: error: Need type annotation for "off"  [var-annotated]
pandas/io/stata.py:687: error: Need type annotation for "val"  [var-annotated]
pandas/io/stata.py:1698: error: Incompatible types in assignment (expression has type "DataFrame", variable has type "ndarray[Any, dtype[Any]]")  [assignment]
pandas/io/stata.py:1700: error: Incompatible types in assignment (expression has type "DataFrame", variable has type "ndarray[Any, dtype[Any]]")  [assignment]
pandas/io/stata.py:1701: error: "ndarray[Any, dtype[Any]]" has no attribute "columns"  [attr-defined]
pandas/io/stata.py:1707: error: "ndarray[Any, dtype[Any]]" has no attribute "index"  [attr-defined]
pandas/io/stata.py:1711: error: Incompatible types in assignment (expression has type "DataFrame", variable has type "ndarray[Any, dtype[Any]]")  [assignment]
pandas/io/stata.py:1711: error: Argument 1 to "_do_select_columns" of "StataReader" has incompatible type "ndarray[Any, dtype[Any]]"; expected "DataFrame"  [arg-type]
pandas/io/stata.py:1721: error: Incompatible types in assignment (expression has type "DataFrame", variable has type "ndarray[Any, dtype[Any]]")  [assignment]
pandas/io/stata.py:1721: error: Argument 1 to "_insert_strls" of "StataReader" has incompatible type "ndarray[Any, dtype[Any]]"; expected "DataFrame"  [arg-type]
pandas/io/stata.py:1725: error: "ndarray[Any, dtype[Any]]" has no attribute "index"  [attr-defined]
pandas/io/stata.py:1730: error: "ndarray[Any, dtype[Any]]" has no attribute "columns"  [attr-defined]
pandas/io/stata.py:1740: error: Incompatible types in assignment (expression has type "DataFrame", variable has type "ndarray[Any, dtype[Any]]")  [assignment]
pandas/io/stata.py:1743: error: Incompatible types in assignment (expression has type "DataFrame", variable has type "ndarray[Any, dtype[Any]]")  [assignment]
pandas/io/stata.py:1743: error: Argument 1 to "_do_convert_missing" of "StataReader" has incompatible type "ndarray[Any, dtype[Any]]"; expected "DataFrame"  [arg-type]
pandas/io/stata.py:1752: error: "ndarray[Any, dtype[Any]]" has no attribute "columns"  [attr-defined]
pandas/io/stata.py:1762: error: Incompatible types in assignment (expression has type "DataFrame", variable has type "ndarray[Any, dtype[Any]]")  [assignment]
pandas/io/stata.py:1763: error: Argument 1 to "_do_convert_categoricals" of "StataReader" has incompatible type "ndarray[Any, dtype[Any]]"; expected "DataFrame"  [arg-type]
pandas/io/stata.py:1783: error: Incompatible types in assignment (expression has type "DataFrame", variable has type "ndarray[Any, dtype[Any]]")  [assignment]
pandas/io/stata.py:1786: error: "ndarray[Any, dtype[Any]]" has no attribute "set_index"  [attr-defined]
pandas/io/stata.py:1786: error: "ndarray[Any, dtype[Any]]" has no attribute "pop"  [attr-defined]
pandas/io/stata.py:1788: error: Incompatible return value type (got "ndarray[Any, dtype[Any]]", expected "DataFrame")  [return-value]
pandas/io/parsers/c_parser_wrapper.py:385: error: Unused "type: ignore" comment
pandas/io/parsers/c_parser_wrapper.py:402: error: Argument 1 to "concatenate" has incompatible type "List[Union[ExtensionArray, ndarray[Any, Any]]]"; expected "Union[_SupportsArray[dtype[Any]], Sequence[_SupportsArray[dtype[Any]]], Sequence[Sequence[_SupportsArray[dtype[Any]]]], Sequence[Sequence[Sequence[_SupportsArray[dtype[Any]]]]], Sequence[Sequence[Sequence[Sequence[_SupportsArray[dtype[Any]]]]]]]"  [arg-type]
pandas/core/series.py:1983: error: Argument "minlength" to "bincount" has incompatible type "Optional[int]"; expected "SupportsIndex"  [arg-type]
pandas/core/generic.py:6983: error: Incompatible types in assignment (expression has type "Index", variable has type "ndarray[Any, dtype[signedinteger[Any]]]")  [assignment]
pandas/core/frame.py:2431: error: Argument 1 to "append" of "list" has incompatible type "Union[type, dtype[Any], str]"; expected "dtype[_SCT]"  [arg-type]
pandas/core/window/rolling.py:359: error: Unused "type: ignore" comment
pandas/core/reshape/reshape.py:1028: error: Incompatible types in assignment (expression has type "ndarray[Any, dtype[signedinteger[Any]]]", variable has type "Index")  [assignment]
pandas/core/reshape/reshape.py:1038: error: Incompatible types in assignment (expression has type "ndarray[Any, dtype[Any]]", variable has type "Index")  [assignment]
pandas/core/reshape/melt.py:136: error: Incompatible types in assignment (expression has type "ndarray[Any, dtype[Any]]", variable has type "Series")  [assignment]
pandas/core/internals/managers.py:1094: error: No overload variant of "__getitem__" of "tuple" matches argument type "ndarray[Any, dtype[signedinteger[Any]]]"  [call-overload]
pandas/core/internals/managers.py:1094: note: Possible overload variants:
pandas/core/internals/managers.py:1094: note:     def __getitem__(self, int) -> Block
pandas/core/internals/managers.py:1094: note:     def __getitem__(self, slice) -> Tuple[Block, ...]
pandas/core/internals/managers.py:1103: error: Argument "blkno" to "_iset_single" of "BlockManager" has incompatible type "ndarray[Any, dtype[signedinteger[Any]]]"; expected "int"  [arg-type]
pandas/core/internals/managers.py:1130: error: Incompatible types in assignment (expression has type "int", variable has type "ndarray[Any, dtype[signedinteger[Any]]]")  [assignment]
pandas/core/internals/managers.py:1131: error: No overload variant of "__getitem__" of "tuple" matches argument type "ndarray[Any, dtype[signedinteger[Any]]]"  [call-overload]
pandas/core/internals/managers.py:1131: note: Possible overload variants:
pandas/core/internals/managers.py:1131: note:     def __getitem__(self, int) -> Block
pandas/core/internals/managers.py:1131: note:     def __getitem__(self, slice) -> Tuple[Block, ...]
pandas/core/internals/managers.py:1156: error: Non-overlapping container check (element type: "int", container item type: "ndarray[Any, dtype[signedinteger[Any]]]")  [comparison-overlap]
pandas/core/internals/blocks.py:376: error: Argument 1 to "delete" has incompatible type "Union[ndarray[Any, Any], ExtensionArray]"; expected "Union[_SupportsArray[dtype[Any]], Sequence[_SupportsArray[dtype[Any]]], Sequence[Sequence[_SupportsArray[dtype[Any]]]], Sequence[Sequence[Sequence[_SupportsArray[dtype[Any]]]]], Sequence[Sequence[Sequence[Sequence[_SupportsArray[dtype[Any]]]]]]]"  [arg-type]
pandas/core/internals/array_manager.py:1206: error: Need type annotation for "array"  [var-annotated]
pandas/core/groupby/ops.py:163: error: Unused "type: ignore" comment
pandas/core/groupby/ops.py:813: error: Incompatible types in assignment (expression has type "List[<nothing>]", variable has type "ndarray[Any, dtype[signedinteger[Any]]]")  [assignment]
pandas/io/sas/sas7bdat.py:802: error: Incompatible types in assignment (expression has type "Series", target has type "ndarray[Any, dtype[Any]]")  [assignment]
pandas/io/sas/sas7bdat.py:805: error: Incompatible types in assignment (expression has type "Series", target has type "ndarray[Any, dtype[Any]]")  [assignment]
pandas/io/sas/sas7bdat.py:805: error: Argument 1 to "_convert_datetimes" has incompatible type "ndarray[Any, dtype[Any]]"; expected "Series"  [arg-type]
pandas/io/sas/sas7bdat.py:807: error: Incompatible types in assignment (expression has type "Series", target has type "ndarray[Any, dtype[Any]]")  [assignment]
pandas/io/sas/sas7bdat.py:807: error: Argument 1 to "_convert_datetimes" has incompatible type "ndarray[Any, dtype[Any]]"; expected "Series"  [arg-type]
pandas/io/sas/sas7bdat.py:810: error: Incompatible types in assignment (expression has type "Series", target has type "ndarray[Any, dtype[Any]]")  [assignment]
pandas/io/sas/sas7bdat.py:812: error: "ndarray[Any, dtype[Any]]" has no attribute "str"  [attr-defined]
pandas/io/sas/sas7bdat.py:816: error: "ndarray[Any, dtype[Any]]" has no attribute "str"  [attr-defined]
pandas/tests/extension/date/array.py:112: error: "object_" object is not iterable  [misc]
Found 80 errors in 27 files (checked 1340 source files)

[seberg]

In the other PR/issue, this:

E numpy.core._exceptions._UFuncNoLoopError: ufunc 'my_ufunc' did not contain a loop with signature matching types (<class 'numpy.dtype[int64]'>, <class 'numpy.dtype[int64]'>, <class 'numpy.dtype[int64]'>) -> None

was said to be the only "real" issue (not typing related). I think this is a duplicate of numpy/numpy#20699 (comment) probably, I will look into it (I guess there may be non-nativ byte-order involved, although maybe not necessarily always)

[attack68]

fyi: dev-requirements.txt issues numba>=0.50.1, which resolves to the latest version: 0.54.1, which has the specific dependency of numpy<1.21.0, so any package resolver should yield numpy==1.20.3 for the dev environment, which doesn't run mypy without failures. I, at least, need numpy==1.21.0 to get mypy success on all files.

[padrepitufo]

not sure why but versions of numpy below 1.22 are causing whitesource security vulnerabilities triggers to go off per https://www.whitesourcesoftware.com/vulnerability-database/CVE-2021-41496 . It would seem that 1.21.5 would solve it as the fix (numpy/numpy@16f7824) shows it's present in 1.21.5

[seberg]

@padrepitufo we have been getting nonsense CVEs for NumPy, this is one of them. I am pretty disappointed by these CVEs appearing out of no-where and being assigned ridiculous severity levels.

[phofl]

I‘ll prepare a pr adressing the typing issues