Groupby with observed=True doesn't sort

[topper-123]

Code Sample, a copy-pastable example if possible

>>> df = pd.DataFrame({'A': pd.Categorical(['b', 'a']), 'B': [1, 2]})
>>> df.groupby('A').sum()  # ok
   B
A
a  2
b  1
>>> df.groupby('A', observed=True).sum()  # wrong sort order
   B
A
b  1
a  2

Problem description

By default groupby has sort=True, so should sort the groupings, also when using observed=True. In the above example, the sorting doesn't work together with observed=True.

Expected Output

>>> df.groupby('A', observed=True).sum()
   B
A
a  2
b  1

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.6.7.final.0
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 78 Stepping 3, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None
LOCALE: None.None

pandas: 0.24.0
pytest: 4.0.1
pip: 18.1
setuptools: 40.4.2
Cython: None
numpy: 1.15.4
scipy: None
pyarrow: None
xarray: None
IPython: 6.5.0
sphinx: None
patsy: None
dateutil: 2.7.5
pytz: 2018.7
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: None
openpyxl: 2.5.9
xlrd: 1.1.0
xlwt: None
xlsxwriter: None
lxml.etree: None
bs4: None
html5lib: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None
gcsfs: None

[jreback]

closed by #25908