Multi-category sorting misbehaves with missing categories

[jonmmease]

Originally reported in plotly/plotly.py#1491.

There seems to a multi-category sorting bug that wasn't fixed by #3362.

CodePen: https://codepen.io/jonmmease/pen/VNeEqq?editors=1010

var fig = {
  "data": [
    {
      "name": "Fruits",
      "type": "bar",
      "x": [
        [
          2017,
          2017,
          2017,
          2017,
          2017,
          2017,
          2017,
          2017,
          2018,
          2018,
          2018,
          2018
        ],
        [
          4,
          5,
          6,
          8,
          9,
          10,
          11,
          12,
          1,
          2,
          3,
          4
        ]
      ],
      "y": [
        57246.0,
        1587.0,
        63189.0,
        30680.0,
        26071.0,
        117519.0,
        27519.0,
        42643.0,
        8559.0,
        22355.0,
        59363.0,
        62955.0
      ]
    },
    {
      "name": "Price",
      "type": "scatter",
      "x": [
        [
          2017,
          2017,
          2017,
          2017,
          2017,
          2017,
          2017,
          2017,
          2018,
          2018,
          2018,
          2018
        ],
        [
          4,
          5,
          6,
          8,
          9,
          10,
          11,
          12,
          1,
          2,
          3,
          4
        ]
      ],
      "y": [
        0.22991999,
        0.27284184,
        0.18438336,
        0.24866362,
        0.18280848,
        0.19382398,
        0.17268069,
        0.18802617,
        0.17560463,
        0.13415343,
        0.14773512,
        0.31319196
      ],
      "yaxis": "y2"
    }
  ],
  "layout": {
    "height": 600,
    "showlegend": true,
    "yaxis": {
      "showgrid": true,
      "title": {
        "text": "Fruits volume"
      }
    },
    "yaxis2": {
      "anchor": "x",
      "overlaying": "y",
      "showgrid": false,
      "side": "right",
      "title": {
        "text": "Price per unit"
      }
    }
  }
}

Plotly.newPlot('myDiv', fig, {showSendToCloud: true});

The problem is that in the second multi-category (2018), the 4 is sorted before 1, 2, 3 even though they are specified in ascending sorted order.

As noted in plotly/plotly.py#1491, this seems to happen for categories that are not present in the first multi-category (2017).

[etpinard]

I think that's the intended behavior under the default configuration. A priori (forget these are numbers) why should 1 be sorted before 4?

What we should do though is implement categoryorder and categoryarray for multicategory. Ok, if I convert this ticket into a type: feature?

[etpinard]

This patch

https://github.com/plotly/plotly.js/compare/multicategory-sort-questionable-fix

would "fix" this. It does generate a diff in a few of our mocks. I wouldn't be comfortable releasing this in v1 w/o a new attribute.

[alexcjohnson]

It would be a complicated algorithm, but I think we could do better than that:

• sequence the outer level (no change there)
• sequence the inner level separately for each outer value (ie separate by outer value and delete duplicates from each sub-list)
• make a combined list for the inner level that does its best to retain the order of each of these sub-lists:
  • search for an item that's first in any sublist that it appears in
  • if no such item is found, pick the item whose first conflict (it's not the first sublist item) is in the latest sublist
  • pop this item off the sublists and add it to the combined list.
  • repeat until all items have been consumed
• use this combined list to re-order inner-level categories

This would probably still change some of the existing test images, but less so and with any luck we'll be able to deem those changes bugfixes.

[ThibTrip]

Hi, if anyone comes up with a PR for this I wanted to remind him to future-proof the algorithm i.e. plan for more than 2 levels (see #3300 (comment)). Wish I could help but my javascript knowledge is close to zero. Anyways thanks again for putting my issue from plotly.py to plotly.js.

[emmanuelle]

Also reported in https://community.plotly.com/t/multicategory-xaxis-label-arrangement-is-not-proper/37359/3
and the corresponding codepen is https://codepen.io/emmanuelle-plotly/pen/zYGgMjz?editable=true
If I understand well there is no way to force the inner labels to be sorted in ascending order?

[gvwilson]

Hi - we are currently trying to tidy up Plotly's public repositories to help us focus our efforts on things that will help users most. Since this issue has been sitting for several years, I'm going to close it. If it's still a concern, we'd be grateful if you could open a new issue (with a short reproducible example if appropriate) so that we can add it to our backlog. Thanks for your help - @gvwilson