Skip to content

BUG: merge raises for how='outer'/'right' when duplicate suffixes are specified #29697

Closed
@jschendel

Description

@jschendel

Code Sample, a copy-pastable example if possible

On master the following raises for how='outer' and how='right' with duplicate suffixes:

In [1]: import pandas as pd; pd.__version__
Out[1]: '0.26.0.dev0+958.g545d17529'

In [2]: df1 = pd.DataFrame({'A': list('ab'), 'B': [0, 1]})

In [3]: df2 = pd.DataFrame({'A':list('ac'), 'B': [100, 200]})

In [4]: pd.merge(df1, df2, on="A", how="outer", suffixes=("_x", "_x"))
---------------------------------------------------------------------------
ValueError: Buffer has wrong number of dimensions (expected 1, got 0)

In [5]: pd.merge(df1, df2, on="A", how="right", suffixes=("_x", "_x"))
---------------------------------------------------------------------------
ValueError: Buffer has wrong number of dimensions (expected 1, got 0)

Note that above works with how='inner' and how='left':

In [6]: pd.merge(df1, df2, on="A", how="inner", suffixes=("_x", "_x"))
Out[6]: 
   A  B_x  B_x
0  a    0  100

In [7]: pd.merge(df1, df2, on="A", how="left", suffixes=("_x", "_x"))
Out[7]: 
   A  B_x    B_x
0  a    0  100.0
1  b    1    NaN

Likewise, if unique suffixes are specified then how='outer' and how='right' work fine:

In [8]: pd.merge(df1, df2, on="A", how="outer", suffixes=("_x", "_y"))
Out[8]: 
   A  B_x    B_y
0  a  0.0  100.0
1  b  1.0    NaN
2  c  NaN  200.0

In [9]: pd.merge(df1, df2, on="A", how="right", suffixes=("_x", "_y"))
Out[9]: 
   A  B_x  B_y
0  a  0.0  100
1  c  NaN  200

Problem description

pandas.merge raises for how='outer' and how='right' with duplicate suffixes.

Expected Output

I'd expect In [4] and In [5] not to raise and produce output similar to Out[8] and Out[9] but with the duplicate suffix names.

Output of pd.show_versions()

INSTALLED VERSIONS

commit : 545d175
python : 3.7.3.final.0
python-bits : 64
OS : Darwin
OS-release : 18.6.0
machine : x86_64
processor : i386
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 0.26.0.dev0+958.g545d17529
numpy : 1.16.4
pytz : 2019.1
dateutil : 2.8.0
pip : 19.1.1
setuptools : 41.6.0.post20191030
Cython : 0.29.13
pytest : 4.6.2
hypothesis : 4.23.6
sphinx : 1.8.5
blosc : None
feather : None
xlsxwriter : 1.1.8
lxml.etree : 4.3.3
html5lib : 1.0.1
pymysql : None
psycopg2 : None
jinja2 : 2.10.1
IPython : 7.5.0
pandas_datareader: None
bs4 : 4.7.1
bottleneck : 1.2.1
fastparquet : 0.3.0
gcsfs : None
lxml.etree : 4.3.3
matplotlib : 3.1.0
numexpr : 2.6.9
odfpy : None
openpyxl : 2.6.2
pandas_gbq : None
pyarrow : 0.11.1
pytables : None
s3fs : 0.2.1
scipy : 1.2.1
sqlalchemy : 1.3.4
tables : 3.5.2
xarray : 0.12.1
xlrd : 1.2.0
xlwt : 1.3.0
xlsxwriter : 1.1.8

Metadata

Metadata

Assignees

Labels

Needs TestsUnit test(s) needed to prevent regressionsReshapingConcat, Merge/Join, Stack/Unstack, Explodegood first issue

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions