Skip to content

Commit 3c4db90

Browse files
committed
Add docs to investigate footprint to performance docs
1 parent 3bb872e commit 3c4db90

File tree

2 files changed

+25
-19
lines changed

2 files changed

+25
-19
lines changed

docs/contributor/CONTRIBUTING.md

Lines changed: 0 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -299,22 +299,3 @@ mx --env ../../graal/vm/mx.vm/ce \
299299
--jvm-config=native \
300300
--python-vm-config=default --
301301
```
302-
303-
## Finding Memory Leaks
304-
305-
For best performance we keep references to long-lived user objects (mostly functions, classes, and modules) directly in the AST nodes when using the default configuration of a single Python context (as is used when running the launcher).
306-
For better sharing of warm-up and where absolutely best peak performance is not needed, contexts can be configured with a shared engine and the ASTs will be shared across contexts.
307-
However, that implies we *must* not store any user objects strongly in the ASTs.
308-
We test that we have no PythonObjects alive after a Context is closed that are run as part of our JUnit tests.
309-
These can be run by themselves, for example, like so:
310-
311-
```bash
312-
mx python-leak-test --lang python \
313-
--shared-engine \
314-
--code 'import site, json' \
315-
--forbidden-class com.oracle.graal.python.builtins.objects.object.PythonObject \
316-
--keep-dump
317-
```
318-
319-
The `--keep-dump` option will print the heapdump location and leave the file there rather than deleting it.
320-
It can then be opened for example with VisualVM to check for the paths of any leaked object, if there are any.

docs/contributor/INVESTIGATING_PERFORMANCE.md

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,3 +53,28 @@ This helps ensure that expected code is inlined (or not).
5353
When I identify something that takes long using gprofng, for example, I find it useful to check if that stuff is inlined as expected on SVM during the HostInliningPhase.
5454

5555
Supposedly Intel VTune and Oracle Developer Studio work well, but I haven't tried them.
56+
57+
## Memory Usage
58+
59+
Memory usage is best tracked with VisualVM for the Java heap.
60+
For best performance we keep references to long-lived user objects (mostly functions, classes, and modules) directly in the AST nodes when using the default configuration of a single Python context (as is used when running the launcher).
61+
For better sharing of warm-up and where absolutely best peak performance is not needed, contexts can be configured with a shared engine and the ASTs will be shared across contexts.
62+
However, that implies we *must* not store any user objects strongly in the ASTs.
63+
We test that we have no PythonObjects alive after a Context is closed that are run as part of our JUnit tests.
64+
These can be run by themselves, for example, like so:
65+
66+
```bash
67+
mx python-leak-test --lang python \
68+
--shared-engine \
69+
--code 'import site, json' \
70+
--forbidden-class com.oracle.graal.python.builtins.objects.object.PythonObject \
71+
--keep-dump
72+
```
73+
74+
The `--keep-dump` option will print the heapdump location and leave the file there rather than deleting it.
75+
It can then be opened for example with VisualVM to check for the paths of any leaked object, if there are any.
76+
77+
For native code, use native memory profiling tools.
78+
I have used [`massif`](https://valgrind.org/docs/manual/ms-manual.html) in the past to find allocations and memory issues in native extensions, but be aware of the large overhead.
79+
However, once you do find something interesting using `massif`, [`rr`](https://rr-project.org/) is a good option to dive further into it, because then you can break around places massif found allocations, and use memory breakpoints and reverse and forward execution to find where the memory is allocated and released.
80+
This can be useful to identify memory leaks in our C API emulation.

0 commit comments

Comments
 (0)