Skip to content

Different order in datetime-column using sort_values on multiple columns #13230

Closed
@marcomayer

Description

@marcomayer

Code Sample, a copy-pastable example if possible

#0.17.1:
df = pd.DataFrame([1,2,3,4,5], columns=list('A'), index=pd.date_range('2010-01-01 09:00:00', periods=5, freq='s')).reset_index()

df['date'] = df['index']
del df['index']
df.loc[4,'A'] = 4
df.loc[4,'date'] = pd.NaT

print(df.sort_values(['A','date']))

  A                date
0  1 2010-01-01 09:00:00
1  2 2010-01-01 09:00:01
2  3 2010-01-01 09:00:02
4  4                 NaT
3  4 2010-01-01 09:00:03

#0.18.1:
df = pd.DataFrame([1,2,3,4,5], columns=list('A'), index=pd.date_range('2010-01-01 09:00:00', periods=5, freq='s')).reset_index()

df['date'] = df['index']
del df['index']
df.loc[4,'A'] = 4
df.loc[4,'date'] = pd.NaT

print(df.sort_values(['A','date']))

   A                date
0  1 2010-01-01 09:00:00
1  2 2010-01-01 09:00:01
2  3 2010-01-01 09:00:02
3  4 2010-01-01 09:00:03
4  4                 NaT

Expected Output

This one was hard to find. The order stays the same as in 0.17.1 when using only sort_values('date'), but using multiple cols, it changes sorting datetimes with NaT. Couldn't find anything in the Changelogs that points to a reason for this.

output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.5.1.final.0
python-bits: 64
OS: Darwin
OS-release: 15.5.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: de_DE.UTF-8

pandas: 0.18.1
nose: 1.3.7
pip: 8.1.2
setuptools: 20.3
Cython: 0.23.4
numpy: 1.10.4
scipy: 0.17.1
statsmodels: 0.6.1
xarray: None
IPython: 4.2.0
sphinx: 1.3.5
patsy: 0.4.0
dateutil: 2.5.1
pytz: 2016.2
blosc: None
bottleneck: 1.0.0
tables: 3.2.2
numexpr: 2.5.2
matplotlib: 1.5.1
openpyxl: 2.3.2
xlrd: 0.9.4
xlwt: 1.0.0
xlsxwriter: 0.8.4
lxml: 3.6.0
bs4: 4.4.1
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: 1.0.12
pymysql: None
psycopg2: None
jinja2: 2.8
boto: 2.39.0
pandas_datareader: 0.2.1

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions