BUG: Setting values to slice fails with duplicated column name

[leonarduschen]

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• (optional) I have confirmed this bug exists on the master branch of pandas.

---

Originally posted on StackOverflow.

Possibly related to #15695 (the traceback looks different though)

---

Code Sample, a copy-pastable example

import pandas as pd
df = pd.DataFrame(columns=['a', 'b', 'b'])
df.loc[:, 'a'] = list(range(5))  # raise ValueError

Traceback:

Traceback (most recent call last):
  File "c:\Users\leona\pandas\main.py", line 3, in <module>
    df.loc[:, 'a'] = list(range(5))
  File "c:\Users\leona\pandas\pandas\core\indexing.py", line 691, in __setitem__
    iloc._setitem_with_indexer(indexer, value, self.name)
  File "c:\Users\leona\pandas\pandas\core\indexing.py", line 1636, in _setitem_with_indexer
    self._setitem_single_block(indexer, value, name)
  File "c:\Users\leona\pandas\pandas\core\indexing.py", line 1862, in _setitem_single_block
    self.obj._mgr = self.obj._mgr.setitem(indexer=indexer, value=value)
  File "c:\Users\leona\pandas\pandas\core\internals\managers.py", line 565, in setitem
    return self.apply("setitem", indexer=indexer, value=value)
  File "c:\Users\leona\pandas\pandas\core\internals\managers.py", line 428, in apply
    applied = getattr(b, f)(**kwargs)
  File "c:\Users\leon\pandas\pandas\core\internals\blocks.py", line 1022, in setitem
    values[indexer] = value
ValueError: cannot copy sequence with size 5 to array axis with dimension 0

Problem description

It works with no duplicated column:

df = pd.DataFrame(columns=['a', 'b', 'c'])
df.loc[:, 'a'] = list(range(5))

These work even with duplicated column names:

df = pd.DataFrame(columns=['a', 'b', 'b'])
df['a'] = list(range(5))  # Same as expected output below

df = pd.DataFrame(columns=['a', 'b', 'b'])
df.a = list(range(5))  # Same as expected output below

Setting on new column name is okay:

df = pd.DataFrame(columns=['a', 'b', 'b'])
df.loc[:, 'c'] = list(range(5))

#      a    b    b  c
# 0  NaN  NaN  NaN  0
# 1  NaN  NaN  NaN  1
# 2  NaN  NaN  NaN  2
# 3  NaN  NaN  NaN  3
# 4  NaN  NaN  NaN  4

Expected Output

   a    b    b
0  0  NaN  NaN
1  1  NaN  NaN
2  2  NaN  NaN
3  3  NaN  NaN
4  4  NaN  NaN

Output of pd.show_versions()

Output:

INSTALLED VERSIONS

commit : 122d502
python : 3.9.0.final.0
python-bits : 64
OS : Windows
OS-release : 10
Version : 10.0.18362
machine : AMD64
processor : Intel64 Family 6 Model 142 Stepping 11, GenuineIntel
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : English_Singapore.1252

pandas : 1.3.0.dev0+83.g122d50246.dirty
numpy : 1.19.4
pytz : 2020.4
dateutil : 2.8.1
pip : 20.2.3
setuptools : 49.2.1
Cython : 0.29.21
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : None
pandas_datareader: None
bs4 : None
bottleneck : None
fsspec : None
fastparquet : None
gcsfs : None
matplotlib : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
numba : None

[rhshadrach]

Thanks for the report. Further investigations and PRs to fix are most welcome.