Description
Pandas version checks
-
I have checked that this issue has not already been reported.
-
I have confirmed this bug exists on the latest version of pandas.
-
I have confirmed this bug exists on the master branch of pandas.
Reproducible Example
import pandas as pd
import numpy as np
df = pd.DataFrame(np.random.rand(5, 3))
shifted = df.shift(6, axis=1, fill_value=None)
shifted.dtypesIssue Description
As the DataFrame has only 3 columns, shifting with periods=6 produces an mulfunctioning dtypes attribute and error below is shown:
Traceback (most recent call last):
File "/Users/wenjun/miniconda3/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3444, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-11-251a20aaa112>", line 1, in <module>
shifted.dtypes
File "/Users/wenjun/Code/pandas/pandas/core/generic.py", line 5716, in dtypes
data = self._mgr.get_dtypes()
File "/Users/wenjun/Code/pandas/pandas/core/internals/managers.py", line 228, in get_dtypes
return dtypes.take(self.blknos)
File "/Users/wenjun/Code/pandas/pandas/core/internals/managers.py", line 168, in blknos
self._rebuild_blknos_and_blklocs()
File "pandas/_libs/internals.pyx", line 723, in pandas._libs.internals.BlockManager._rebuild_blknos_and_blklocs
IndexError: Out of bounds on buffer access (axis 0)
The same code works under pandas 1.3.x.
I don't know if it is right to submit issues for a working branch, but this issue seems exists for a long time.
Expected Behavior
shifted.dtypes shall display a same Series as df.dtypes, or a ValueError shall be raised when periods out of bounds are provided.
Installed Versions
INSTALLED VERSIONS
commit : 3e9f09f
python : 3.9.5.final.0
python-bits : 64
OS : Darwin
OS-release : 21.1.0
Version : Darwin Kernel Version 21.1.0: Wed Oct 13 17:33:23 PDT 2021; root:xnu-8019.41.5~1/RELEASE_X86_64
machine : x86_64
processor : i386
byteorder : little
LC_ALL : None
LANG : None
LOCALE : zh_CN.UTF-8
pandas : 1.4.0.dev0+1432.g3e9f09fa63
numpy : 1.21.2
pytz : 2021.3
dateutil : 2.8.2
pip : 21.1.3
setuptools : 52.0.0.post20210125
Cython : 0.29.24
pytest : 6.2.4
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 3.0.2
IPython : 7.29.0
pandas_datareader: None
bs4 : None
bottleneck : 1.3.2
fsspec : None
fastparquet : None
gcsfs : None
matplotlib : None
numexpr : 2.7.3
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : 4.0.1
pyxlsb : None
s3fs : None
scipy : 1.7.1
sqlalchemy : 1.4.27
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
numba : None