memory leak after plotting?

[choldgraf]

I've noticed that after leaving interactive 3d plots open for a long time, the browser starts to run really, really slowly.

This might be on the JS side, but is there a function in python for clearing the cache / javascript memory / etc? Usually my browser will crash after I make about 10 trisurf plots, and I have to restart it as well as my python kernel.

Anybody else experience this?

[choldgraf]

note - this is plotting in offline mode, with pretty large 3d surfaces (60,000 triangles or so). It works fine for the first few plots, however.

[jackparmer]

FYI plotly.js WebGL folks @mikolalysenko @etpinard @monfera ^^^

@choldgraf We've seen slow memory leaks from WebGL plots in the past, though not in a while. Thanks for reporting!

[choldgraf]

hey folks - was anybody able to replicate this problem?

[monfera]

@choldgraf Hello Chris, have you a minimal example, e.g. specific type of plot? Also, how long does this issue generally take to manifest, and is it the specific browser tab that gets slow (slow to rotate/pan I guess?) or the entire browser, i.e. all its tabs? Which browser are you using, recent Chrome? Lastly, what hardware is it sitting on, especially, is it integrated graphics or dedicated video, and possibly, the size of the GPU memory?

[choldgraf]

Sorry @monfera , I should have been more explicit in my original post! I'm getting this problem when making trisurf plots, which is calling Mesh3d under the hood, I believe. I'm plotting a large amount of triangles, ~60K or so. I can try to put together some simple code to re-create the problem, though it's not going to be super clear-cut since there's not an error or anything.

[monfera]

@choldgraf thank you for the information! I've been wrapping up a functional addition to the WebGL based plotting. As soon as it's done, fixing memory leaks is planned to be the next item.

[choldgraf]

Sounds great, I appreciate the help. I'd open a PR if I was efficient at all with javascript, sorry to ask you to do it.

In a sort-of related question, do you know how optimized the webgl plotting scripts are? I'm trying to think if there are any low-hanging fruit for speeding up some of the heavier 3D visualizations.

[monfera]

Currently we have a somewhat linear data flow that recalculates more than what's needed if there's incremental data. Besides the memory leak issue, the other item I'm scheduled to work on next is incremental performance measurements and enhancements. The two tasks have a good bit of overlap because both issues involve profiling and identification of improvement points. I trust there'll be quite a few low hanging fruits but it's hard to speculate. One possibility is using typed float arrays upstream (from the moment of receiving the data) so as to increase efficiency and possibly reduce intermediate memory allocations, and another relates to speeding up data transfer into plotly.js, but I'd rather not pinpoint anything before solid profiling and some proof of concept.

[choldgraf]

Currently we have a somewhat linear data flow that recalculates more than what's needed if there's incremental data.

Ah, sounds like that might be related to some slowdown on plotting large datasets I've been experiencing (specifically again with Mesh3d, but I haven't really tried it with other plot types)

The two tasks have a good bit of overlap because both issues involve profiling and identification of improvement points. I trust there'll be quite a few low hanging fruits but it's hard to speculate.

I recently took a similar pass on the python 3d trisurf library, and there were a lot of really easy fixes that sped things up by a factor of 10+, so hopefully something similar for JS :)

Please let me know if there's anything I could help with...not super experienced with JS but perhaps I could help benchmarking in python or something

[monfera]

Thanks Chris for your offer! Since it's good to consider 'in vivo' use cases, it would actually be quite helpful to use a minimalistic case that still fairly characterizes the speed requirements and has as few parts as possible, e.g. constraining it into a single HTML + JS file where the HTML file links to the plotly script CDN URL and the JS code generates a (random, or mathematical) mesh, i.e. no actual data payload is necessary. Of course the Javascript calls var beginsAt = Performance.now(); [slow code]; var endsAt = Performance.now(); would exclude the large input generation.

We have examples on codepen where the whole thing is the most basic HTML file and a JS file with very simple invocation, not unlike the Python API call, e.g.

• http://codepen.io/etpinard/pen/YWwNxo (simplest thing)
• http://codepen.io/etpinard/pen/WxrRoGand (procedurally generates test data, and performs rotation)

Data generation can be as simple as making a surface for a grid where z[i] = Math.sin(i) + Math.cos(i); as long as it generates roughly the same amount of work for plotly.js and representative of what you want to see speed up.

[choldgraf]

Cool - is it cool if I wrote something in Python? That's partially because
the data transfer to JS also seems to be a potential point of speedup.

In this case, I could probably just simulate a sphere w/ the same number of
triangle faces as the brains that I've been plotting...that sound alright?

On Mon, Jun 13, 2016 at 1:04 PM, Robert Monfera notifications@github.com
wrote:

Thanks Chris for your offer! Since it's good to consider 'in vivo' use
cases, it would actually be quite helpful to use a minimalistic case that
still fairly characterizes the speed requirements and has as few parts as
possible, e.g. constraining it into a single HTML + JS file where the HTML
file links to the plotly script CDN URL and the JS code generates a
(random, or mathematical) mesh, i.e. no actual data payload is necessary.
Of course the Javascript calls var beginsAt = Performance.now(); [slow
code]; var endsAt = Performance.now(); would exclude the large input
generation.

We have examples on codepen where the whole thing is the most basic HTML
file and a JS file with very simple invocation, not unlike the Python API
call, e.g.

• http://codepen.io/etpinard/pen/YWwNxo (simplest thing)
• http://codepen.io/etpinard/pen/WxrRoGand (procedurally generates
  test data, and performs rotation)

Data generation can be as simple as making a surface for a grid where z[i]
= Math.sin(i) + Math.cos(i); as long as it generates roughly the same
amount of work for plotly.js and representative of what you want to see
speed up.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#487 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/ABwSHWCrAVO0A6uNWkxHhCxo2z609uInks5qLbe-gaJpZM4IpKnV
.

[monfera]

@choldgraf Yes it sounds good, thank you! The simpler the code, the better (no other Python libs etc.) because I'll probably transcribe it into JS for quick iterations. If you see that something doesn't impact performance, it can be omitted. E.g. if the absence of a face color array doesn't speed up processing, then it can be ignored. Also it's helpful if you can share some measurement, e.g. measuring time in python, or in JS with Performance.now(), to have a rough guide of time ratio spent with interfacing vs. inside plotly.js, but don't worry about it if it's too tedious, I can do that too.

[choldgraf]

OK - is it alright if I put it in a jupyter notebook for the python side of
things? That way you could use %lprun etc to get performance by lines and
such...

On Mon, Jun 13, 2016 at 3:11 PM, Robert Monfera notifications@github.com
wrote:

@choldgraf https://github.com/choldgraf Yes it sounds good, thank you!
The simpler the code, the better (no other Python libs etc.) because I'll
probably transcribe it into JS for quick iterations. If you see that
something doesn't impact performance, it can be omitted. E.g. if the
absence of a face color array doesn't speed up processing, then it can be
ignored. Also it's helpful if you can share some measurement, e.g.
measuring time in python, or in JS with Performance.now(), to have a rough
guide of time ratio spent with interfacing vs. inside plotly.js, but
don't worry about it if it's too tedious, I can do that too.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#487 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/ABwSHQvRk3ZKQDihhvrFcPifUuaikYV5ks5qLdWUgaJpZM4IpKnV
.

[monfera]

Perfect, works great e.g. in a github repo file!

[choldgraf]

gimme 5 minutes :)

On Mon, Jun 13, 2016 at 3:18 PM, Robert Monfera notifications@github.com
wrote:

Perfect, works great e.g. in a github repo file!

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#487 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/ABwSHcaz_CyLnXkxgC0EhpDWxh5NJ3jcks5qLdcsgaJpZM4IpKnV
.