BUG: _NDFrameIndexer._setitem_with_indexer() doesn't handle MultiIndex and mixed types properly

[seth-p]

Compare the behavior of the homogeneously typed panel1 with that of the heterogeneously types panel2:

In [1]: import pandas as pd

In [29]: panel1 = pd.Panel([[[0., 0.]], [[0., 0.]]], items=['A', 'B'], minor_axis=['x', 'y'])

In [30]: panel2 = pd.Panel([[[0., 0.]], [['o', 'o']]], items=['A', 'B'], minor_axis=['x', 'y'])

In [33]: panel1['C'] = 0.

In [34]: panel2['C'] = 0.

In [35]: panel1.dtypes
Out[35]:
A    float64
B    float64
C    float64
dtype: object

In [36]: panel2.dtypes
Out[36]:
A     object
B     object
C    float64
dtype: object

In [37]: panel1.loc['C', 0, :] = [5, 6]

In [38]: panel1['C']
Out[38]:
   x  y
0  5  6

In [39]: panel2.loc['C', 0, :] = [5, 6]
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-39-57fe49e884e4> in <module>()
----> 1 panel2.loc['C', 0, :] = [5, 6]

C:\Python34\lib\site-packages\pandas\core\indexing.py in __setitem__(self, key, value)
    116     def __setitem__(self, key, value):
    117         indexer = self._get_setitem_indexer(key)
--> 118         self._setitem_with_indexer(indexer, value)
    119
    120     def _has_valid_type(self, k, axis):

C:\Python34\lib\site-packages\pandas\core\indexing.py in _setitem_with_indexer(self, indexer, value)
    460
    461                     if len(labels) != len(value):
--> 462                         raise ValueError('Must have equal len keys and value '
    463                                          'when setting with an iterable')
    464

ValueError: Must have equal len keys and value when setting with an iterable

In [41]: pd.show_versions()

INSTALLED VERSIONS
------------------
commit: None
python: 3.4.3.final.0
python-bits: 64
OS: Windows
OS-release: 7
machine: AMD64
processor: Intel64 Family 6 Model 58 Stepping 9, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None

pandas: 0.16.1
nose: 1.3.6
Cython: 0.22
numpy: 1.9.1
scipy: None
statsmodels: None
IPython: 3.1.0
sphinx: None
patsy: 0.3.0
dateutil: 2.4.2
pytz: 2015.4
bottleneck: 0.8.0
tables: 3.1.1
numexpr: 2.4
matplotlib: 1.4.3
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: 1.0.5
pymysql: None
psycopg2: None

Pandas 0.16.2 exhibits the same error.

[jreback]

we'll call this a bug, though would have to step thru. I think its taking an incorrect path. Note that panels are essentially a dict of frames, so changing the lowest dimension might need to coerce most/all of the panel.

[Garrett-R]

Yeah, looks like a bug to me. @jreback, I'm a bit confused though. In seth-p's output above, we have:

In [36]: panel2.dtypes
Out[36]:
A     object
B     object
C    float64
dtype: object

But isn't it nonsense that it outputs the types of the "items", shouldn't we be talking about the types of the "minor axis" (aka columns)?

[jreback]

@Garrett-R .dtypes returns the block dtypes, you are probably looking for this

In [10]: p = tm.makePanel()

In [11]: p
Out[11]: 
<class 'pandas.core.panel.Panel'>
Dimensions: 3 (items) x 30 (major_axis) x 4 (minor_axis)
Items axis: ItemA to ItemC
Major_axis axis: 2000-01-03 00:00:00 to 2000-02-11 00:00:00
Minor_axis axis: A to D

In [12]: p._data
Out[12]: 
BlockManager
Items: Index([u'ItemA', u'ItemB', u'ItemC'], dtype='object')
Axis 1: DatetimeIndex(['2000-01-03', '2000-01-04', '2000-01-05', '2000-01-06', '2000-01-07', '2000-01-10', '2000-01-11', '2000-01-12', '2000-01-13', '2000-01-14', '2000-01-17', '2000-01-18', '2000-01-19',
               '2000-01-20', '2000-01-21', '2000-01-24', '2000-01-25', '2000-01-26', '2000-01-27', '2000-01-28', '2000-01-31', '2000-02-01', '2000-02-02', '2000-02-03', '2000-02-04', '2000-02-07',
               '2000-02-08', '2000-02-09', '2000-02-10', '2000-02-11'],
              dtype='datetime64[ns]', freq='B', tz=None)
Axis 2: Index([u'A', u'B', u'C', u'D'], dtype='object')
FloatBlock: slice(0, 3, 1), 3 x 30 x 4, dtype: float64

In [13]: p.apply(lambda x: x.dtype)
Out[13]: 
     ItemA    ItemB    ItemC
A  float64  float64  float64
B  float64  float64  float64
C  float64  float64  float64
D  float64  float64  float64

In [14]: p.apply(lambda x: x.dtype, axis='items')
Out[14]: 
                  A        B        C        D
2000-01-03  float64  float64  float64  float64
2000-01-04  float64  float64  float64  float64
2000-01-05  float64  float64  float64  float64
2000-01-06  float64  float64  float64  float64
2000-01-07  float64  float64  float64  float64
2000-01-10  float64  float64  float64  float64
2000-01-11  float64  float64  float64  float64
2000-01-12  float64  float64  float64  float64
2000-01-13  float64  float64  float64  float64
2000-01-14  float64  float64  float64  float64
2000-01-17  float64  float64  float64  float64
2000-01-18  float64  float64  float64  float64
2000-01-19  float64  float64  float64  float64
2000-01-20  float64  float64  float64  float64
2000-01-21  float64  float64  float64  float64
2000-01-24  float64  float64  float64  float64
2000-01-25  float64  float64  float64  float64
2000-01-26  float64  float64  float64  float64
2000-01-27  float64  float64  float64  float64
2000-01-28  float64  float64  float64  float64
2000-01-31  float64  float64  float64  float64
2000-02-01  float64  float64  float64  float64
2000-02-02  float64  float64  float64  float64
2000-02-03  float64  float64  float64  float64
2000-02-04  float64  float64  float64  float64
2000-02-07  float64  float64  float64  float64
2000-02-08  float64  float64  float64  float64
2000-02-09  float64  float64  float64  float64
2000-02-10  float64  float64  float64  float64
2000-02-11  float64  float64  float64  float64

[seth-p]

By the way, I get this error when items is a MultiIndex even when all dtypes are float64:

In [486]: import pandas as pd

In [487]: import numpy as np

In [488]: p1 = pd.Panel(0., items=['A', 'B'], major_axis=[0, 1], minor_axis=['X', 'Y' ,'Z'],
                        dtype=np.float64)

In [489]: p2 = pd.Panel(0., items=pd.MultiIndex.from_tuples([('A','a'), ('B','b')]),
                        major_axis=[0, 1], minor_axis=['X', 'Y' ,'Z'], dtype=np.float64)

In [490]: p1.iloc[0, 0, :] = [1, 2, 3]

In [491]: p2.iloc[0, 0, :] = [1, 2, 3]
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-491-89a2908fbf6b> in <module>()
----> 1 p2.iloc[0, 0, :] = [1, 2, 3]

C:\Python34\lib\site-packages\pandas\core\indexing.py in __setitem__(self, key, value)
    113     def __setitem__(self, key, value):
    114         indexer = self._get_setitem_indexer(key)
--> 115         self._setitem_with_indexer(indexer, value)
    116
    117     def _has_valid_type(self, k, axis):

C:\Python34\lib\site-packages\pandas\core\indexing.py in _setitem_with_indexer(self, indexer, value)
    468
    469                     if len(labels) != len(value):
--> 470                         raise ValueError('Must have equal len keys and value '
    471                                          'when setting with an iterable')
    472

ValueError: Must have equal len keys and value when setting with an iterable

In [492]: p1.dtypes
Out[492]:
A    float64
B    float64
dtype: object

In [493]: p2.dtypes
Out[493]:
A  a    float64
B  b    float64
dtype: object

[seth-p]

Any chance someone can fix this for release 0.17? It makes modifying a Panel that uses a MultiIndex very cumbersome.