REGR: DataFrame.shift with periods>len(columns) GH#44978 (#45005)

Author: jbrockmendel

doc/source/whatsnew/v1.4.0.rst

@@ -878,6 +878,7 @@ Other
 - Bug in :meth:`Series.replace` and :meth:`DataFrame.replace` with ``value=None`` and ExtensionDtypes (:issue:`44270`,:issue:`37899`)
 - Bug in :meth:`FloatingArray.equals` failing to consider two arrays equal if they contain ``np.nan`` values (:issue:`44382`)
 - Bug in :meth:`DataFrame.shift` with ``axis=1`` and ``ExtensionDtype`` columns incorrectly raising when an incompatible ``fill_value`` is passed (:issue:`44564`)
+- Bug in :meth:`DataFrame.shift` with ``axis=1`` and ``periods`` larger than ``len(frame.columns)`` producing an invalid :class:`DataFrame` (:issue:`44978`)
 - Bug in :meth:`DataFrame.diff` when passing a NumPy integer object instead of an ``int`` object (:issue:`44572`)
 - Bug in :meth:`Series.replace` raising ``ValueError`` when using ``regex=True`` with a :class:`Series` containing ``np.nan`` values (:issue:`43344`)
 - Bug in :meth:`DataFrame.to_records` where an incorrect ``n`` was used when missing names were replaced by ``level_n`` (:issue:`44818`)

pandas/core/internals/managers.py

@@ -388,12 +388,13 @@ def shift(self: T, periods: int, axis: int, fill_value) -> T:
             # GH#35488 we need to watch out for multi-block cases
             # We only get here with fill_value not-lib.no_default
             ncols = self.shape[0]
+            nper = abs(periods)
+            nper = min(nper, ncols)
             if periods > 0:
                 indexer = np.array(
-                    [-1] * periods + list(range(ncols - periods)), dtype=np.intp
+                    [-1] * nper + list(range(ncols - periods)), dtype=np.intp
                 )
             else:
-                nper = abs(periods)
                 indexer = np.array(
                     list(range(nper, ncols)) + [-1] * nper, dtype=np.intp
                 )

pandas/tests/frame/methods/test_shift.py

@@ -664,3 +664,15 @@ def test_shift_axis1_categorical_columns(self):
             columns=ci,
         )
         tm.assert_frame_equal(result, expected)
+
+    @td.skip_array_manager_not_yet_implemented
+    def test_shift_axis1_many_periods(self):
+        # GH#44978 periods > len(columns)
+        df = DataFrame(np.random.rand(5, 3))
+        shifted = df.shift(6, axis=1, fill_value=None)
+
+        expected = df * np.nan
+        tm.assert_frame_equal(shifted, expected)
+
+        shifted2 = df.shift(-6, axis=1, fill_value=None)
+        tm.assert_frame_equal(shifted2, expected)