scattergl can't resolve date with sub-minute precision

[etpinard]

See http://codepen.io/etpinard/pen/ORjbkR

Probably because time stamps that long can't fit on the GPU. We'll probably have to rescale to data arrays about the xy axis range.

[monfera]

@etpinard I see sub-minute precision (in this case, seconds) on both versions in the codepen example:

[etpinard]

This is what I'm seeing

The top figure (which is a scattergl trace) has no drawn marker points.

[monfera]

Indeed, I initially misinterpreted it as a tick label rendering issue. In this case I'd look into #413 first, because it's useful to solve this issue in combination with making the non-fancy scattergl work with dates, and then consider rolling the date resolution approach to the fancy version.

[monfera]

@jackparmer a question for you at the end: having a PR for #413, we now have a PoC solution for this issue. The root cause was that the data path from the plotly.js through gl-vis to the GPU is of a 32 bit resolution, and

new Date('2016-10-10').valueOf() i.e. time from the epoch as an integral multiple of milliseconds is much larger than Math.pow(2, 24)

I broadened the pipeline to 64 bits up till the (32-bit) WebGL renderer, where an offset is introduced. This works well for arbitrary start dates (e.g. in 2016) as long as the span of time domain is not too long, for example, 10/29/2016 to 10/30/2016 can be represented with very fine resolution. But the current PoC can't cover e.g. 2016 ... 2026 with a fine temporal resolution, as we still have a 32 bit (actually 24 bit) representation. Examples:

• At an exact millisecond resolution, this covers 4.66 hours (2 ** 24) / 1000 / 3600
• At a second resolution, this covers half a year (*)
• At a minute resolution, this covers 32 years (*)

(*) with the last two examples, there's some dithering - eg. at the second resolution, maybe we don't get an exact 10/10/2016 21:58:47 but more like 10/10/2016 21:58:47.23429 - which may not matter in practice if values are shown with precision that's limited to second (I don't think we automatically have such resolution-aware decimal rounding though).

This 32 bit resolution limit is native to WebGL. With more work, it'd be possible to pass on two 32 bit arrays and combine them in the vertex shader. It slows down the vertex shader a bit, though it's unlikely to be a bottleneck. Still, there might be unexpected complications.

I'd add that this resolution limitation isn't inherent to dates/times. It applies for plain numbers as well.

What do we know about the expected combination of the span of time domain and the expected temporal resolution, should we shoot for a software based 64 bit solution in the shader or let's get feedback first?

[monfera]

Example: millisecond resolution over one second

[etpinard]

Slightly unrelated, but looking at the above screenshot, we should try to address gl2d date axis ticks formatting as discussed here.

[monfera]

@etpinard @jackparmer doubling the available WebGL precision has worked out, giving us a millisecond-level resolution even over years or millennia: gl-vis/gl-scatter2d#1

gl-vis/gl-scatter2d-fancy and its constituents are also covered.

[etpinard]

done in #1079