Managing Off-Heap Memory

This topic explains how to manage off-heap memory in VMware Tanzu GemFire.

You can configure Tanzu GemFire to store region values in off-heap memory, memory within the JVM that is not subject to Java garbage collection.

Garbage collection (GC) within a JVM can prove to be a performance impediment. A server cannot exert control over when garbage collection within the JVM heap memory takes place, and the server has little control over the triggers for invocation. Off-heap memory offloads values to a storage area that is not subject to Java GC. By taking advantage of off-heap storage, an application can reduce the amount of heap storage that is subject to GC overhead.

Off-heap memory works in conjunction with the heap, it does not replace it. The keys are stored in heap memory space. Tanzu GemFire’s own memory manager handles the off-heap memory with better performance than the Java garbage collector would for certain sets of region data.

The resource manager monitors the contents of off-heap memory and invokes memory management operations in accordance with two thresholds similar to those used for monitoring the JVM heap: eviction-off-heap-percentage and critical-off-heap-percentage.

On-heap and Off-heap Objects

The following objects are always stored in the JVM heap:

Region metadata
Entry metadata
Keys
Indexes
Subscription queue elements

The following objects can be stored in off-heap memory:

Values - maximum value size is 2GB
Reference counts
List of free memory blocks
WAN queue elements

Note: Do not use functional range indexes with off-heap data, as they are not supported. An attempt to do so generates an exception.

Off-heap Recommendations

Off-heap storage is best suited to data patterns where:

Stored values are relatively uniform in size
Stored values are mostly less than 128K in size
The usage patterns involve cycles of many creates followed by destroys or clear
The values do not need to be frequently deserialized
Many of the values are long-lived reference data

Be aware that Tanzu GemFire has to perform extra work to access the data stored in off-heap memory since it is stored in serialized form. This extra work may cause some use cases to run slower in an off-heap configuration, even though they use less memory and avoid garbage collection overhead. However, even with the extra deserialization, off-heap storage may give you the best performance. Features that may increase overhead include

frequent updates
stored values of widely varying sizes
deltas
queries

Implementation Details

The off-heap memory manager is efficient at handling region data values that are all the same size or are of fixed sizes. With fixed and same-sized data values allocated within the off-heap memory, freed chunks can often be re-used, and there is little or no need to devote cycles to defragmentation.

Region values that are less than or equal to eight bytes in size will not reside in off-heap memory, even if the region is configured to use off-heap memory. These very small size region values reside in the JVM heap in place of a reference to an off-heap location. This performance enhancement saves space and load time.

Controlling Off-heap Use with the Resource Manager

The Tanzu GemFire resource manager controls off-heap memory by means of two thresholds, in much the same way as it does JVM heap memory. See Using the Tanzu GemFire Resource Manager. The resource manager prevents the cache from consuming too much off-heap memory by evicting old data. If the off-heap memory manager is unable to keep up, the resource manager refuses additions to the cache until the off-heap memory manager has freed an adequate amount of memory.

The resource manager has two threshold settings, each expressed as a percentage of the total off-heap memory. Both are deactivated by default.

Eviction Threshold. The percentage of off-heap memory at which eviction should begin. Evictions continue until the resource manager determines that off-heap memory use is again below the eviction threshold. Set the eviction threshold with the eviction-off-heap-percentage region attribute. The resource manager enforces an eviction threshold only on regions with the HEAP_LRU characteristic. If critical threshold is non-zero, the default eviction threshold is 5% below the critical threshold. If critical threshold is zero, the default eviction threshold is 80% of total off-heap memory.

The resource manager enforces eviction thresholds only on regions whose LRU eviction policies are based on heap percentage. Regions whose eviction policies based on entry count or memory size use other mechanisms to manage evictions. See Eviction for more detail regarding eviction policies.
Critical Threshold. The percentage of off-heap memory at which the cache is at risk of becoming inoperable. When cache use exceeds the critical threshold, all activity that might add data to the cache is refused. Any operation that would increase consumption of off-heap memory throws a LowMemoryException instead of completing its operation. Set the critical threshold with the critical-off-heap-percentage region attribute.

Critical threshold is enforced on all regions, regardless of LRU eviction policy, though it can be set to zero to deactivate its effect.

Specifying Off-heap Memory

To use off-heap memory, specify the following options when setting up servers and regions:

Start the JVM as described in Tuning the JVM’s Garbage Collection Parameters. In particular, set the initial and maximum heap sizes to the same value. Sizes less than 32GB are optimal when you plan to use off-heap memory.
From gfsh, start each server that will support off-heap memory with a non-zero off-heap-memory-size value, specified in megabytes (m) or gigabytes (g). If you plan to use the resource manager, specify critical threshold, eviction threshold, or (in most cases) both.

Example:
```
gfsh> start server --name=server1 -–initial-heap=10G -–max-heap=10G -–off-heap-memory-size=200G \
-–lock-memory=true -–critical-off-heap-percentage=90 -–eviction-off-heap-percentage=80
```
Mark regions whose entry values should be stored off-heap by setting the off-heap region attribute to true Configure other region attributes uniformly for all members that host data for the same region. .

Example:
```
gfsh>create region --name=region1 --type=PARTITION_HEAP_LRU --off-heap=true
```

gfsh Off-heap Support

gfsh supports off-heap memory in server and region creation operations and in reporting functions:

alter disk-store: --off-heap=(true | false) resets the off-heap attribute for the specified region. See alter disk-store for details.
create region: --off-heap=(true | false) sets the off-heap attribute for the specified region. See create region for details.
describe member: Displays off-heap size.
describe offline-disk-store: Shows if an off-line region is off-heap.
describe region: Displays the value of a region’s off-heap attribute.
show metrics: Includes off-heap metrics maxMemory, freeMemory, usedMemory, objects, fragmentation and defragmentationTime.
start server: Supports off-heap options --lock-memory, ‑‑off-heap-memory-size, ‑‑critical-off-heap-percentage, and ‑‑eviction-off-heap-percentage See start server for details.

ResourceManager API

The org.apache.geode.cache.control.ResourceManager interface defines methods that support off-heap use:

public void setCriticalOffHeapPercentage(float Percentage)
public float getCriticalOffHeapPercentage()
public void setEvictionOffHeapPercentage(float Percentage)
public float getEvictionOffHeapPercentage()

The gemfire.properties file supports one off-heap property:

off-heap-memory-size: Specifies the size of off-heap memory in megabytes (m) or gigabytes (g). For example:

off-heap-memory-size=4096m
off-heap-memory-size=120g

See gemfire.properties and gfsecurity.properties (Tanzu GemFire Properties) for details.

The cache.xml file supports one region attribute:

off-heap(=true | false): Specifies that the region uses off-heap memory; defaults to false. For example:

<region-attributes
  off-heap="true">
</region-attributes>

See <region-attributes> for details.

The cache.xml file supports two resource manager attributes:

critical-off-heap-percentage=value: Specifies the percentage of off-heap memory at or above which the cache is considered in danger of becoming inoperable due to out of memory exceptions. See <resource-manager> for details.

eviction-off-heap-percentage=value: Specifies the percentage of off-heap memory at or above which eviction should begin. Can be set for any region, but actively operates only in regions configured for HEAP_LRU eviction. See <resource-manager> for details.

For example:

<cache>
...
  <resource-manager 
    critical-off-heap-percentage="99.9" 
    eviction-off-heap=-percentage="85"/>
  ...
</cache>

Tuning Off-heap Memory Usage

Tanzu GemFire collects statistics on off-heap memory usage which you can view with the gfsh show metrics command. See Off-Heap (OffHeapMemoryStats) for a description of available off-heap statistics.

Off-heap memory is optimized, by default, for storing values of 128 KB in size. This figure is known as the “maximum optimized stored value size,” which we will denote here by maxOptStoredValSize. If your data typically runs larger, you can enhance performance by increasing the OFF_HEAP_FREE_LIST_COUNT system parameter to a number larger than maxOptStoredValSize/8, where maxOptStoredValSize is expressed in KB (1024 bytes). So, the default values correspond to:

128 KB / 8 = (128 * 1024) / 8 = 131,072 / 8 = 16,384
-Dgemfire.OFF_HEAP_FREE_LIST_COUNT=16384

To optimize for a maximum optimized stored value size that is twice the default, or 256 KB, the free list count should be doubled:

-Dgemfire.OFF_HEAP_FREE_LIST_COUNT=32768

During the tuning process, you can toggle the off-heap region attribute on and off, leaving other off-heap settings and parameters in place, in order to compare your application’s on-heap and off-heap performance.