How Delta Propagation Works

This topic explains how delta propagation works in VMware Tanzu GemFire.

Delta propagation reduces the amount of data you send over the network. You do this by sending only the change, or delta, information about an object, instead of sending the entire changed object. If you do not use cloning when applying the deltas, you can also expect to generate less garbage in your receiving JVMs.

In most distributed data management systems, the data stored in the system tends to be created once and then updated frequently. These updates are sent to other members for event propagation, redundancy management, and cache consistency in general. Tracking only the changes in an updated object and sending only the deltas mean lower network transmission costs and lower object serialization/deserialization costs. Performance improvements can be significant, especially when changes to an object are small relative to its overall size.

Tanzu GemFire propagates object deltas using methods that you program. The methods are in the org.apache.geode.Delta interface, which you implement in your cached objects’ classes. If any of your classes are plain old Java objects, you need to wrap them for this implementation.

This figure shows delta propagation for a change to an entry with key, k, and value object, v.

1. get operation. The get works as usual: the cache returns the full entry object from the local cache or, if it is not available there, from a remote cache or from a loader.
2. update methods. You need to add code to the object’s update methods so that they save delta information for object updates, in addition to the work they were already doing.
3. put operation. The put works as usual in the local cache, using the full value, then calls hasDelta to see if there are deltas and toDelta to serialize the information. Distribution is the same as for full values, according to member and region configuration.
4. receipt of delta at remote member. fromDelta extracts the delta information that was serialized by toDelta and applies it to the object in the local cache. The delta is applied directly to the existing value or to a clone, depending on how you configure it for the region.
5. additional distributions. As with full distributions, receiving members forward the delta according to their configurations and connections to other members. For example, if VM1 is a client and VM2 is a server, VM2 forwards the delta to its peers and its other clients as needed. Receiving members do not recreate the delta; toDelta is only called in the originating member.

General Characteristics of Delta Propagation

To use the delta propagation feature, all updates on a key in a region must have value types that implement the org.apache.geode.Delta interface. You cannot mix object types for an entry key where some of the types implement delta and some do not. This is because, when a type implementing the Delta interface is received for an update, the existing value for the key is cast to a Delta type to apply the received delta. If the existing type does not also implement the Delta interface, the operation throws a ClassCastException.

Note: Only the object itself being placed in the cache can implement the Delta interface and propagate changes. Any sub-objects of the cache object do not propagate their changes.

Sometimes fromDelta cannot be invoked because there is no object to apply the delta to in the receiving cache. When this happens, the system automatically does a full value distribution to the receiver. These are the possible scenarios:

1. If the system can determine beforehand that the receiver does not have a local copy, it sends the initial message with the full value. This is possible when regions are configured with no local data storage, such as with the region shortcut settings PARTITION_PROXY and REPLICATE_PROXY. These configurations are used to accomplish things like provide data update information to listeners and to pass updates forward to clients.
2. In less obvious cases, such as when an entry has been locally deleted, first the delta is sent, then the receiver requests a full value and that is sent. Whenever the full value is received, any further distributions to the receiver’s peers or clients uses the full value.

Tanzu GemFire also does not propagate deltas for:

• Transactional commit
• The putAll operation
• JVMs running Tanzu GemFire versions that do not support delta propagation (6.0 and earlier)

Supported Topologies and Limitations

The following topologies support delta propagation (with some limitations):

• Peer-to-peer. Tanzu GemFire system members distribute and receive entry changes using delta propagation, with these requirements and caveats:
  • Regions must be partitioned or have their scope set to distributed-ack or global. The region shortcut settings for distributed regions use distributed-ack scope. Delta propagation does not work for regions with distributed-no-ack scope because the receiver could not recover if an exception occurred while applying the delta.
  • For partitioned regions, if a receiving peer does not hold the primary or a secondary copy of the entry, but still requires a value, the system automatically sends the full value.
  • To receive deltas, a region must be non-empty. The system automatically sends the full value to empty regions. Empty regions can send deltas.
• Client/server. Tanzu GemFire clients can always send deltas to the servers, and servers can usually sent deltas to clients. These configurations require the servers to send full values to the clients, instead of deltas:
  • When the client’s gemfire.properties setting conflate-events is set to true, the servers send full values for all regions.
  • When the server region attribute enable-subscription-conflation is set to true and the client gemfire.properties setting conflate-events is set to server, the servers send full values for the region.
  • When the client region is configured with the PROXY client region shortcut setting (empty client region), servers send full values.

• Multi-site (WAN). If a domain object implements the Delta interface, delta updates to it will be sent between sites. The ideal use case for this feature is to have each site start with the same entries so that when deltas are sent from one site to the other, they can be applied directly.

  If a delta is being processed by a GatewayReceiver in a remote site and either of these cases are true, then the full value will be sent by the originating GatewaySender:

  • The remote site does not have the value to which the delta must be applied

  • The domain object’s fromDelta method throws an InvalidDeltaException

    In both of those cases, an InvalidDeltaException for that key will be sent back to the originating site, and an event containing the full value for that key will be enqueued at the back of the GatewaySender’s queue. Any deltas received after that full value event is processed by the remote site will be applied to that value.

    Note: If the domain object’s fromDelta method throws any other kind of exception, the full value will not be sent.

  The full value will always be sent in these cases even if the delta is available:

  • If either the GatewaySender or GatewayReceiver is running in a member whose version of GemFire earlier than v10.0
  • If the member running either the GatewaySender or GatewayReceiver configures delta-propagation=false
  • If the GatewaySender has conflation enabled
  • If the partitioned region processing the deltas is attached to a GatwaySender configured with parallel=false
  • Any update event that contains delta bytes but is treated like a create event (e.g. if the entry was destroyed while the update was occurring)
  • Any event that occurs before the GatewaySender has a connection to the GatewayReceiver (since the connection is required to establish the GatewayReceiver’s GemFire version)
  • If the GatewaySender starts first, it will not have a connection to the GatewayReceiver. Any events created in that state will have the full value because the remote site version is unknown at that time. Similarly, if the GatewaySender starts first, followed by the GatewayReceiver, and the GatewaySender is then paused before any operations occur, no connection is established and all events will be sent as full values (as the GemFire version of the GatewayReceiver’s member is unknown at the time the operations are put into the Gateway queue).