Tuning of the garbage collector
This page discusses the effects of the Java Virtual Machine’s garbage collector with regards to Neo4j performance. In this setting, the heap is separated into an old generation and a young generation, while new objects are allocated in the young generation, and then later moved to the old generation, if they stay live (in use) for long enough.
When a generation fills up, the garbage collector performs a collection, during which all other threads in the process are paused. The young generation is quick to collect since the pause time correlates with the live set of objects. In the old generation, pause times roughly correlates with the size of the heap. For this reason, the heap should ideally be sized and tuned such that transaction and query state never makes it to the old generation.
The heap size is configured with the server.memory.heap.initial_size (in MBs) setting in the neo4j.conf file.
The initial size of the heap is specified by the server.memory.heap.initial_size setting, or with the -Xms???m flag, or chosen heuristically by the JVM itself if left unspecified.
The JVM will automatically grow the heap as needed, up to the maximum size.
The growing of the heap requires a full garbage collection cycle.
It is recommended to set the initial heap size and the maximum heap size to the same value.
This way the pause that happens when the garbage collector grows the heap can be avoided.
If the new generation is too small, short-lived objects may be moved to the old generation too soon. This is called premature promotion and will slow the database down by increasing the frequency of old generation garbage collection cycles. If the new generation is too big, the garbage collector may decide that the old generation does not have enough space to fit all the objects it expects to promote from the new to the old generation. This turns new generation garbage collection cycles into old generation garbage collection cycles, again slowing the database down. Running more concurrent threads means that more allocations can take place in a given span of time, in turn increasing the pressure on the new generation in particular.
|
The Compressed OOPs feature in the JVM allows object references to be compressed to use only 32 bits.
The feature saves a lot of memory but is only available for heaps up to 32 GB.
The maximum applicable size varies from platform and JVM version.
The |
How to tune the specific garbage collection algorithm depends on both the JVM version and the workload. It is recommended to test the garbage collection settings under realistic load for days or weeks. Problems like heap fragmentation can take a long time to surface.
To gain good performance, these are the things to look into first:
-
Make sure the JVM is not spending too much time performing garbage collection. The goal is to have a large enough heap to make sure that heavy/peak load will not result in so called GC-trashing. Performance can drop as much as two orders of magnitude when GC-trashing happens. Having too large heap may also hurt performance so you may have to try some different heap sizes.
-
Neo4j needs enough heap memory for the transaction state and query processing, plus some head-room for the garbage collector. As heap memory requirements are so workload-dependent, it is common to see heap memory configurations from 1 GB, up to 32 GB.
Edit the following properties:
| Property Name | Meaning |
|---|---|
|
initial heap size (in MB) |
|
maximum heap size (in MB) |
|
additional literal JVM parameter |