VMware vSphere Replication 6.5.1 Release Notes

VMware vSphere Replication 6.5.1.5 | 11 FEB 2021 | Build 17483533 VMware vSphere Replication 6.5.1.4 | 27 JUNE 2019 | Build 14002386 VMware vSphere Replication 6.5.1.3 | 03 MAY 2018 | Build 8136506 VMware vSphere Replication 6.5.1.2 | 30 NOV 2017 | Build 7184801 VMware vSphere Replication 6.5.1.1 | 19 OCT 2017 | Build 6876646 VMware vSphere Replication 6.5.1 | 27 JULY 2017 | Build 6128267 Check for additions and updates to these release notes.

For information about the vSphere Replication 6.5.1.x patch releases, see the corresponding section.

• vSphere Replication 6.5.1.5 Express Patch Release
• vSphere Replication 6.5.1.4 Express Patch Release
• vSphere Replication 6.5.1.3 Express Patch Release
• vSphere Replication 6.5.1.2 Express Patch Release
• vSphere Replication 6.5.1.1 Express Patch Release

 

What's in the Release Notes

These release notes cover the following topics:

• Localization
• What's New
• Product Documentation
• vCloud Suite Licensing and Integration
• Installation
• Upgrading vSphere Replication
• Operational Limits of vSphere Replication
• Open Source Components
• Caveats and Limitations
• Available Patch Releases
• Known Issues

 

Localization

VMware vSphere Replication 6.5.1 is available in the following languages:

• English
• French
• German
• Japanese
• Korean
• Spanish
• Simplified Chinese
• Traditional Chinese

 

What's New

VMware vSphere Replication 6.5.1 provides the following new features:

• vSphere Replication 6.5.1 is compatible with VMware vSphere 6.5 Update 1.
• Supports upgrade migration path from vCenter Server Virtual Appliance 6.0 Update 3 to vCenter Server Virtual Appliance 6.5 Update 1 by delivering a direct upgrade path from vSphere Replication 6.1.2 to vSphere Replication 6.5.1.
• vSphere Replication 6.5.1 now supports the following external databases:
  • Microsoft SQL Server 2014 Service Pack 2
  • Microsoft SQL Server 2016 Service Pack 1
• vSphere Replication 6.5.1 now supports the following guest operating systems:
  • Windows Server 2016
  • CentOS 6.9
  • RHEL 7.3.5
  • Ubuntu 17.04 non Long Term Support (LTS) 

Product Documentation

In addition to the current release notes, you can use the documentation set for vSphere Replication 6.5 that includes the following deliverables.

• vSphere Replication 6.5 Documentation Center
• Interoperability Pages for vSphere Replication 6.5

The documentation set for VMware vCloud Air – Disaster Recovery includes the following deliverables.

• vCloud Air – Disaster Recovery Release Notes
• vSphere Replication for Disaster Recovery to Cloud
• vCloud Air – Disaster Recovery User's Guide

 

vCloud Suite Licensing and Integration

You can license vSphere Replication individually or as part of vCloud Suite. You should consider the licensing and integration options that are available to you.

Some vCloud Suite components are available as standalone products that are licensed on a per-virtual machine basis. When the products are part of vCloud Suite, they are licensed on a per-CPU basis. You can run unlimited number of virtual machines on CPUs that are licensed with vCloud Suite.

You can combine the features of vSphere Replication with other components of vCloud Suite to leverage the full capabilities of the software-defined data center. For more information, see vCloud Suite Architecture Overview and Use Cases.

 

Installation

Download the vSphere Replication .iso image and mount it. You can deploy the vSphere Replication appliance by using the Deploy OVF wizard in the vSphere Web Client. Navigate to the \bin directory in the .iso image and use the corresponding OVF file:

1. vSphere_Replication_OVF10.ovf: Use this file to install all vSphere Replication components, including the vSphere Replication Management Server and a vSphere Replication Server.
2. vSphere_Replication_AddOn_OVF10.ovf: Use this file to install an optional additional vSphere Replication Server.

For more information on installation, see section Installing vSphere Replication in the vSphere Replication Documentation Center.

NOTE: For vCenter Server to vCenter Server replications, the version of the vSphere Replication Management server on the source and the target site must match.

 

Upgrading vSphere Replication

The downloadable ISO image is the only means of upgrading from vSphere Replication 6.1.2 or 6.5 to vSphere Replication 6.5.1. You cannot upgrade vSphere Replication from version 6.1.2 or 6.5 to version 6.5.1 by using vSphere Update Manager or the official VMware Update Repository from the VAMI of the vSphere Replication appliance. See the interoperability pages for further information on supported versions.

Important: Before you initiate an upgrade, verify that the vSphere Replication appliance has an OVF environment, or context. See Checking and Restoring the OVF Context of the vSphere Replication Appliance (2106709).

Verify that you read section Upgrade under Known Issues.

See Upgrade vSphere Replication by Using the Downloadable ISO Image for the procedure on upgrading to vSphere Replication 6.5.1.

 

Operational Limits for vSphere Replication

The operational limits of vSphere Replication 6.5.1 are documented in the VMware Knowledge Base. See Operational Limits for vSphere Replication 6.x (KB 2102453).

Note: vSphere Replication requires additional configuration to support more than 500 replications per a vSphere Replication Management server. See Operational Limits for vSphere Replication 6.х and Configuring Upgraded vSphere Replication Appliances to Support up to 2000 Replications.

 

Open Source Components

The copyright statements and licenses applicable to the open source software components distributed in vSphere Replication 6.5.1 are available at the vSphere Replication Open Source Disclosure page.

 

Caveats and Limitations of vSphere Replication 6.5.1

To ensure successful virtual machine replication, you must verify that your virtual infrastructure respects certain limits before you start the replication.

• vSphere Replication 6.5.1 requires and fully supports vCenter Server 6.5 Update 1.
• The 5 minute RPO scales to a maximum supported limit of 50 VMs on a provisional VVOL datastore.
• vSphere Replication does not support VSS quiescing on Virtual Volumes.
• vSphere Replication cannot replicate virtual machines that share vmdk files in this release.
• vSphere Replication does not support vSphere APIs for IO Filtering on both the source and the target sites. You cannot replicate a virtual machine that is assigned a VM Storage Policy that contains IOFilters, nor can you assign such a policy to the replication target VM. Before configuring a virtual machine for replication, verify that the VM Storage Policy that is assigned to it does not contain IOFilters. Do not assign VM Storage policies with IOFilters to virtual machines that are already configured for replication.
• Deploying more than one vSphere Replication appliance produces a warning on the boot screen. This requires user confirmation to either continue and configure all replications again or shut down the new appliance so that it does not interfere with the old one. This situation does not occur when deploying more than one vSphere Replication servers.
• Each vSphere Replication Management Server can manage a maximum of 2000 replicated virtual machines. See Configuring Upgraded vSphere Replication Appliances to Support up to 2000 Replications (KB 2102463) and Requirements to the Environment... (KB 2107869).
• If you move a virtual machine with replicated disks larger than 2032GB to an ESXi 5.1 or earlier host, vSphere Replication cannot replicate or power on the virtual machine.
• vSphere Replication supports a maximum disk size of 62TB. If you attempt to enable replication on a virtual machine with a disk larger than 62TB, the virtual machine will not perform any replication operation and will not power on.
• vSphere Replication tracks larger blocks on disks over 2TB. Replication performance on a disk over 2TB might be different than replication performance on a disk under 2TB for the same workload depending on how much of the disk goes over the network for a particular set of changed blocks.
• vSphere Replication no longer supports IBM DB2 as the vSphere Replication database, in accordance with the removal of support for DB2 as a supported database for vCenter Server 5.5. If you use DB2 as an external vSphere Replication database, contact VMware support for instructions about how to migrate your data to a supported database.
• vSphere Replication does not support upgrading the VMware Tools package in the vSphere Replication appliance.
• vSphere Replication supports replicating RDMs in Virtual Compatibility Mode. RDMs in Physical Compatibility Mode cannot be configured for replication.
• vSphere Replication does not replicate virtual machine snapshot hierarchy at the target site.
• You can configure virtual machines that are powered off for replication. However, actual replication traffic begins when the virtual machine is powered on.
• When using Storage DRS at a replication site, ensure that you have homogeneous host and datastore connectivity to prevent Storage DRS from performing resource consuming cross-host moves (changing both the host and the datastore) of replica disks.
• For replications to cloud, a seed vApp can be used for only one replication.
• The 5 minute RPO requires the source host to be ESXi 6.0 or later for VSAN, and ESXi 6.5 for other supported datastores.
• To use the network isolation feature, vSphere Replication requires the host to be ESXi 6.0 or later.

Supported Browser Versions

For supported browser versions for the vSphere Web Client, see the documentation of the vSphere Web Client that you use.

 

Available Patch Releases

vSphere Replication 6.5.1.5 Express Patch Release

Released 11 FEB 2021 | Build 17483533

The vSphere Replication 6.5.1.5 Express Patch Release provides important security fixes.

Installation Notes

The downloadable ISO image is the only means of upgrading from vSphere Replication 6.5.1.x and 6.1.2.x to vSphere Replication 6.5.1.5. See the Upgrading vSphere Replication in vSphere Replication Administration for instructions about updating vSphere Replication 6.5.1.x.

You upgrade vSphere Replication by using a downloadable ISO image.

 

vSphere Replication 6.5.1.4 Express Patch Release

Released 27 JUNE 2019 | Build 14002386

The vSphere Replication 6.5.1.4 Express Patch Release provides bug fixes.

 

Installation Notes

The downloadable ISO image is the only means of upgrading from vSphere Replication 6.5.1.x and 6.1.2.x to vSphere Replication 6.5.1.4. See the Upgrading vSphere Replication in vSphere Replication Administration for instructions about updating vSphere Replication 6.5.1.x.

You upgrade vSphere Replication by using a downloadable ISO image.

 

vSphere Replication 6.5.1.3 Express Patch Release

Released 03 MAY 2018 | Build 8136506

• The vSphere Replication 6.5.1.3 Express Patch Release is compatible with VMware vSphere 6.5 Update 2.
• The vSphere Replication 6.5.1.3 Express Patch Release provides a fix to protecting the vSphere Replication appliance itself by using VMware vSphere HA.
• The vSphere Replication 6.5.1.3 Express Patch Release introduces vmware-gss-support package for monitoring network traffic and replaces the tcpdump package. To use tcpdump, open the remote console of your vSphere Replication virtual machine and run /etc/vmware/gss-support/install.sh to install the tcpdump package.

 

Installation Notes

The downloadable ISO image is the only means of upgrading from vSphere Replication 6.5.1.x and 6.1.2.x to vSphere Replication 6.5.1.3. See the Upgrading vSphere Replication in vSphere Replication Administration for instructions about updating vSphere Replication 6.5.1.x.

You upgrade vSphere Replication by using a downloadable ISO image.

 

vSphere Replication 6.5.1.2 Express Patch Release

Released 30 NOV 2017 | Build 7184801

• The vSphere Replication 6.5.1.2 Express Patch Release provides critical security fixes.
• The vSphere Replication 6.5.1.2 Express Patch Release provides Open Source Software (OSS) updates.
• The vSphere Replication 6.5.1.2 Express Patch Release applies the fix for CVE- 2017-1000112.

Installation Notes

The downloadable ISO image is the only means of upgrading from vSphere Replication 6.5.1.x and 6.1.2.x to vSphere Replication 6.5.1.2. See the Upgrading vSphere Replication in vSphere Replication Administration for instructions about updating vSphere Replication 6.5.1.x.

You upgrade vSphere Replication by using a downloadable ISO image.

vSphere Replication 6.5.1.1 Express Patch Release

Released 19 OCT 2017 | Build 6876646

The vSphere Replication 6.5.1.1 Express Patch Release updates the available RPO range for replications from on-premises vSphere environments to cloud when using vCloud Availability for vCloud Director 2.0. In the Configure Replication wizard you can select an RPO range from 5 minutes to 24 hours.

Resolved Issues

The following issues have been fixed in this release.

• vSphere Replication appliance fails to update the VM network port group

  After changing the VM network port group of the vSphere Replication appliance, it continues to use the old port group.

  The issue is fixed in vSphere Replication 6.5.1.4.

• Target VMDK objects are created by using the host-level storage policy, instead of the configured storage policy

  When you initially configure a VM replication and select a storage policy for the target VMDK, the target VMDK object is always created by using the host-level storage policy, regardless of what you select.

  The issue is fixed in vSphere Replication 6.5.1.4.

• vSphere Replication Upgrade interrupts unexpectedly 

  The vSphere Replication appliance is running on a host that is part of a vSAN cluster with vSphere HA enabled. When you power off the host, vSphere HA successfully restarts the appliance on a difference host. As a result the /opt/vmware/etc/vami/ovfEnv.xml inside the appliance becomes blank, which might break the appliance upgrade.

  The issue is fixed in vSphere Replication 6.5.1.4.

• vSphere Replication runs out of memory with an error

  After you configure replications, the vSphere Replication appliance might continue to create threads which results in the following error:

   javax.xml.ws.WebServiceException: java.lang.OutOfMemoryError: unable to create new native thread

  This issue is fixed in vSphere Replication 6.5.1.4. 

• After upgrade to vSphere Replication 6.5.x, the blue Virtual Appliance Management Interface (VAMI) screen in the console does not appear

  After you upgrade the vSphere Replication appliance from version 6.1.1 to version 6.5.x, the blue VAMI screen in the console does not appear.

  This issue is fixed.

• Configuring a replication might fail if the target site has multiple vCenter Server instances in the same Platform Services Controller with vSphere Replication 6.5 or vSphere Replication 8.0 installed

  If you have a configuration where the source site is in a standalone Platform Services Controller (PSC) with a single vCenter Server, and the target site is in a standalone PSC with two or more vCenter Server instances that all have vSphere Replication installed, when you try to configure replication from the source to the target site, the replication might fail. The replication succeeds if the remote vCenter Server the user is logged in is the first registered in the PSC, and fails otherwise.

  The issue is fixed.

• The hbrsrv service fails to start and the system log file contains a corruption error

  vSphere Replication service reaches a limitation of the threads it can start and cannot create threads to execute any more tasks. vSphere Replication fails to start with Error in `/usr/bin/hbrsrv-bin': double free or corruption (out) in /var/log/messages.

  The issue is fixed.

Known Issues

The known issues are grouped as follows.

• Upgrade
• General
• Replications to vCenter Server
• Cloud Replications

Upgrade

• vSphere Replication fails to upgrade from version 6.1.2 to version 6.5.1.3

  The upgrade of vSphere Replication 6.1.2 to version 6.5.1.3 fails with the following error:

  Failed to install updates(Error while running installation tests)

  Workaround: Upgrade vSphere Replication 6.1.2 to version 6.1.2.2 before upgrading to version 6.5.1.3.

• The vSphere Replication Management service does not start after the upgrade

  After you upgrade vSphere Replication, the vSphere Replication Management (VRM) service appears as stopped in the VAMI, and the /opt/vmware/hms/logs/hms-configtool.log file in the virtual appliance contains java.net.ConnectException: Connection refused error messages.

  This problem is observed if the upgrade procedure of the embedded DB schema fails because the vPostgreSQL service was not fully started.

  Workaround:

  1. In the virtual appliance console, log in as the root user.
  2. Run the following command: $ /opt/vmware/hms/bin/hms-configtool -cmd upgrade -configfile /opt/vmware/hms/conf/hms-configuration.xml
     

     The DB schema upgrade starts.

  3. Wait for the DB upgrade procedure to complete.
  4. In the vSphere Replication VAMI, navigate to the Configuration tab, and complete the SSO registration of the appliance.
• Missing vSphere Replication permissions after upgrading the vSphere Replication appliance, certificate or IP change

  If you upgrade the vSphere Replication appliance, or if for some other reason the certificate or the IP address of the vSphere Replication appliance changes, the permissions that are assigned to the default VRM user roles are deleted.
  This problem is observed every time the vSphere Replication extension is unregistered and registered with the vCenter Server extension manager.

  Workaround: Clone the predefined VRM roles and create your custom roles before upgrading the vSphere Replication appliance, or changing its certificate or IP address. The permissions that are assigned to custom roles are not removed.

• The vSphere Replication Virtual Appliance Management Interface (VAMI) becomes inaccessible after upgrade

  After the upgrade, the vSphere Replication VAMI changes and you cannot access it in the same browser window that you used before the upgrade.

  Workaround: Do one of the following.

  • Change the browser that you use to open the VAMI.
  • Close the entire browser and open a new browser window to connect to the VAMI.
  • Clear the cache of your browser.
    

• The vSphere Replication appliance changes to a vSphere Replication server after the upgrade

  If you do not check the OVF context of the vSphere Replication appliance before you perform an upgrade, and the upgrade operation does not fail, the upgraded vSphere Replication appliance appears as a vSphere Replication Server. The data about replications that were configured before the upgrade is lost.

  Workaround:

  • If you have a pre-upgrade snapshot of the vSphere Replication appliance, revert to that snapshot and see Checking and Restoring the OVF Environment of the vSphere Replication Appliance (2106709).
  • If you do not have a pre-upgrade snapshot of the vSphere Replication appliance, uninstall the upgraded vSphere Replication instance and perform a fresh deployment. See Installing and Uninstalling vSphere Replication.

• After an upgrade of the vCenter Server and vSphere Replication, configuring the SSO in the vSphere Replication VAMI fails with error Bad exit code: 1

  After you upgrade the vCenter Server to version 6.5 and vSphere Replication to version 6.5, you must register the appliance with vCenter Single Sign-on. On the Configuration tab of the vSphere Replication VAMI, you enter the LookupService address and the credentials of an SSO administrator, and click Save and Restart Service. The following error message appears: Bad exit code: 1.
  This problem is observed because the upgraded vCenter Server changes its IP address or certificate, but the vSphere Replication Management server preserves the old IP address and certificate of the vCenter Server in its OVF environment. As a result, the validation of the vCenter Server fails.

  Workaround: In the vSphere Web Client, right-click the vSphere Replication Management server VM and power it off and on. This operation forces the update of the OVF environment on the vSphere Replication Management server VM.
  

• Site Recovery Manager cannot be upgraded after upgrading vSphere Replication

  On upgrade of vSphere Replication to version 6.5, Site Recovery Manager cannot be upgraded as the vSphere Replication versions is detected as incompatible. Under solutions manager in vCenter, the vSphere Replication version appears not to have been upgraded though the appliance reports the upgrade is successful.

  Workaround: Register the vSphere Replication appliance with vCenter Single Sign-On.

  1. Connect to the VAMI interface of the vSphere Replication appliance by using a supported browser.
  2. On the Configuration tab, enter the user name and password of an SSO administrator.
     Note: The text boxes for the SSO credentials will not be visible if you are using an unsupported browser.

General

• vSphere Replication shows inconsistent information about status and number of replications or Site Recovery Manager test failover fails 

  When vSphere Replication service reaches a limitation of the threads it can start and cannot create threads to process new events, one of the following can occur:

  • vSphere Replication shows mismatching information about the status and number of replications on the source and target sites.
  • In Site Recovery Manager, test failover fails with the following record in the system log: VR synchronization failed for VRM group ifulgv002a. A generic error occurred in the vSphere Replication Management Server. Exception details: 'com.vmware.hms.replication.sync.DeltaAbortedException.

  Workaround: Change the vSphere Replication configuration to remove the maximum thread limitation.

  1. Log into the vSphere Replication appliance as root.
  2. Open the file /etc/systemd/system.conf in an editor.
  3. Uncomment the line DefaultTasksMax=512 and set its value to DefaultTasksMax=infinity.
  4. Save the file.
  5. Follow the steps provided in KB 2150650.
• Configuring a replication that uses seeds on a VVOL target datastore succeeds, but the replication is in Error state.

  If you configure a replication to use as a seed a VM that has snapshots, the configure operation succeeds, but the replication goes into the Error state at the end of the Initial Full Sync. An issue with a similar error description appears:
  "A replication error occurred at the vSphere Replication Server for replication 'vmname'. Details: 'Error for (datastoreUUID: "vvol:9148a6192d0349de-94149524b5f52bc4"), (diskId: "RDID-fd3ed4de-2356-43c7-a0e2-7bc07a7da012"), (hostId: "host-33"), (pathname: "vmname/vmname.vmdk"), (flags: retriable): Class: NFC Code: 10; NFC error: NFC_DISKLIB_ERROR (Input/output error); Set error flag: retriable; Can't write (multiEx) to remote disk; Can't write (multi) to remote disk'."

  Workaround: Delete the snapshots from the seed VM.

• Some vSphere Replication operations fail or become unresponsive

  If you initiate dozens of vSphere Replication operations simultaneously, some of them might hang. This might also cause the VRMS site to become unresponsive as a result of the vSphere Replication appliance requiring more memory to perform many operations in parallel.

  Workaround:

  1. SSH to the vSphere Replication appliance.
  2. Stop the vcta service:
     service vmware-vcd stop
     Note: Stopping this service interrupts the replications to and from the cloud.

• During full synchronization vSphere Replication fails with error: A replication error occurred at the vSphere Replication Server.

  During full synchronization vSphere Replication might fail with the following error.
  A replication error occurred at the vSphere Replication Server for replication <group_name>. Details: 'Error for (datastoreUUID: "..."), (diskId: "..."), (hostId: "..."), (pathname: "..."), (flags: retriable, pick-new-host, nfc-no-memory): Class: NFC Code: 5; NFC error: NFC_NO_MEMORY; Set error flag: nfc-no-memory; Code set to: Host unable to process request.; Set error flag: retriable; Set error flag: pick-new-host; Can't write (single) to remote disk'.
  
  Usually, this error is transient and the operation succeeds after some time.

• A replication configured with seeds left over from a previous replication on a vSphere Virtual Volumes datastore displays in Not active state

  If you unconfigure a replication after configuring it with seed disks on vSphere Virtual Volumes, the leftover disks are read-only snapshots and cannot be used as seeds for a new replication.

  Workaround: Manually delete the seed disks and reconfigure the replication.

• vSphere Replication UI does not warn the user if the selected target datastore is not compliant with the default datastore policy

  When you configure a VM for replication, the Target Location page has the default datastore policy as the preselected VM Storage Policy. No warning is shown if a storage that is non-compliant with its policy is selected. Depending on the type of non-compliance the replication configuration might fail or succeed. The replicated VM can be recovered, but will later fail to power on due to non-compliance.

  Workaround: Select a concrete policy, and then select a datastore from the Compatible group in the list.

• Replacing the SSL certificate of vCenter Server causes certificate validation errors in vSphere Replication

  If you replace the SSL certificate on the vCenter Server system, a connection error occurs when vSphere Replication attempts to connect to vCenter Server.

  Workaround: For information about how to update vCenter Server certificates and allow solutions such as vSphere Replication to continue to function, see http://kb.vmware.com/kb/2109074.

• Data synchronization fails and the log file of the source vSphere Replication Management Server contains error DeltaAbortedException

  If your environment experiences connectivity issues during data synchronization, you might observe the following problems.

  • Replication group synchronizations fail and the hms<n>.log file in the vSphere Replication Management server at the source site contains the following error message:
    DeltaAbortedException.
  • In Site Recovery Manager, replication group synchronizations fail with the following error message:
    VR synchronization failed for VRM group <group_name>. A generic error occurred in the vSphere Replication Management Server. Exception details: 'com.vmware.hms.replication.sync.DeltaAbortedException' .

  Workaround: Resolve the connectivity issues in your environment before you proceed.

• The initial configuration task fails with error InvalidArgument

  If you configure a replication for a virtual machine that contains disks without UUID, vSphere Replication assigns UUIDs for these disks during the initial configuration. However, if these disks have parent disks, for example preceding snapshots, vSphere Replication cannot assign UUIDs for them and the initial configuration task fails with error InvalidArgument.

  Workaround: Consolidate the disks on the source virtual machine and try configuring a replication again.

• Failover with "Sync latest changes" might fail with SocketTimeoutException when multiple replications are recovered concurrently and there is a huge accumulated delta since the latest synchronization

  The vSphere Replication Management server might not receive due responses through the vCenter reverse proxy when there is heavy replication traffic at the same network. Some replication management or monitoring operations might fail with the following error message:
  'com.vmware.vim.vmomi.client.exception.ConnectionException: java.net.SocketTimeoutException: Read timed out'

  Workaround: Configure network traffic isolation for vSphere Replication traffic, so that the management communication between vCenter and the vSphere Replication Management server is not affected by the heavy replication traffic. See Isolating the Network Traffic of vSphere Replication.

• Virtual machines that are located in the target folder are overwritten during recovery
  

  If the target folder contains a registered virtual machine with the same name as the replicated virtual machine, the registered virtual machine is overwritten during the recovery. When you start the Recovery wizard, vSphere Replication checks the target folder and displays a dialog box for you to confirm the overwrite operation. On rare occasions, after the target check is complete, and while the wizard is still open, a virtual machine might be registered to the target folder. On these occasions, the virtual machine that was copied to the target folder will be overwritten without further notice.

  Workaround: None.

• Replications appear in Not Active (RPO violation) status after changing the IP address of the vSphere Replication server at the target site
  

  If the IP address of the vSphere Replication server at the target site changes, the status of all replications to this site turns to Not Active (RPO violation). This problem is observed because replications on the source site are not reconfigured automatically when the IP address changes.

  Workaround: Reconfigure all replications, so that the source hosts use the new IP address of the target vSphere Replication server.

• Transient Error state during the initial full synchronization

  During the initial synchronization, you might observe that the state of the synchronization changes temporarily to Error and back to normal multiple times. The error state might indicate resource deficiency at the target site. If the IO workload caused by the sync operation is higher than the load that target hosts can handle, the state of the replication will turn to Error. When the IO workload decreases, the error disappears.

  Workaround: Reduce the value of the host configuration option called HBR.TransferMaxContExtents on each ESXi host where replication source VMs are running. The default value is 8, and a lower value decreases the size of data blocks that are sent during one sync update, but increases the duration of the initial full sync. After the initial full sync, change the value back to its default (8) to achieve maximum RPO performance. If transient errors continue to appear during delta synchronizations, it might mean that a lot of changed blocks are transferred during each delta, and the hosts at the target site cannot accommodate the incurred IO workload. In such cases, keep the value of the HBR.TransferMaxContExtents configuration option low.
  Alternatively, you can add more hosts to the secondary site.

• Users that are assigned the VRM administrator or VRM virtual machine replication role cannot access the Configure Replication wizard

  The Configure Replication wizard is not launched if a user that is assigned the predefined VRM administrator or VRM virtual machine replication role logs in the vSphere Web Client and attempts to configure a replication.

  Workaround: Clone the default role to add the Profile-driven storage -> Profile-driven storage view privilege to it, and assign the cloned role to the user.

• Clicking the Configure link for an embedded VR Server does not open the VAMI

  If you navigate to the Replication Servers view, select an embedded vSphere Replication server, and invoke the Configure action, the VAMI is not displayed.

  Workaround: Open a browser and enter the address https://<VR_Appliance_Address>:5480 to open the VAMI.

• The option to enable quiescing is disabled in Configure Replication wizard for a powered off replication source VM, though the guest OS supports quiescing

  For both Linux and Windows sources, the Enable Quiescing option is enabled based on the information about the guest OS. If a virtual machine has never been powered on, ESXi hosts always report no support for quiescing, because the guest OS information is not available.

  Workaround: Verify that replication source VMs have been powered on at least once before you configure replications.

• vSphere Replication service is inaccessible after vCenter Server certificate changes

  If the vCenter Server certificate changes, vSphere Replication becomes inaccessible.

  Workaround: See vSphere Replication is Inaccessible After Changing vCenter Server Certificate.

• vSphere Replication Management Server (VRMS) might leak a partially recovered virtual machine in the target vCenter Server after a failed recovery

  In rare cases VRMS might stop during recovery immediately after registering the recovered virtual machine in the target vCenter Server. The last recovery error in the replication details panel says VRM Server was unable to complete the operation. When VRMS restarts, it cleans up the files for the partially recovered virtual machine. In some cases, it fails to unregister the virtual machine from the target vCenter Server. Subsequent recovery attempts show an error in the recovery wizard that the selected virtual machine folder already contains an entity with the same name.

  Workaround: Manually remove the virtual machine from the target vCenter Server, but keep its disks as they point to the replica placeholder files.

• During replication of multiple virtual machines, a vSphere Replication server might enter a state where it does not accept any further VRMS connections but continues to replicate virtual machines

  Workaround: Reboot the vSphere Replication server.

• vSphere Replication operations fail with a Not Authenticated error

  If you start an operation on one site, for example configuring vSphere Replication on a virtual machine, and then restart vCenter Server and the vSphere Replication appliance on the other site, vSphere Replication operations can fail with the error VRM Server generic error. Please check the documentation for any troubleshooting information. The detailed exception is: 'com.vmware.vim.binding.vim.fault.NotAuthenticated'. This problem is caused by the fact that the vSphere Replication server retains in its cache the connection session from before you restarted vCenter Server and the vSphere Replication appliance.

  Workaround: Clear the vSphere Replication connection cache by logging out of the vSphere Web Client and logging back in again.

• Operation in vSphere Replication Management Server fails with error "... UnmarshallException"

  When the vSphere Replication Management Server experiences high load or transient network errors, operations can fail with UnmarshallException due to errors in the communication layer.

  Workaround: Try the failed operation again.

• The VAMI might not respond when you install an update

  When you upgrade vSphere Replication, a status message 'Installing Updates' might not disappear even after the updates install successfully because the VAMI is not responding.

  Workaround: Refresh the VAMI UI in the browser or open it in a new tab.

• A virtual machine recovered in vSphere Replication does not power on in vCenter Server

  When you use vSphere Replication to run a recovery on a virtual machine, it fails, and the status of the replication is not 'Recovered'. The virtual machine is registered in the vCenter inventory, but when you try to power it on, it fails with error: File [datastorename] path/vmname.vmx was not found. The virtual machine registration as part of the vSphere Replication recovery workflow can succeed in vCenter Server, but the response might not reach the vSphere Replication Management Server due to a transient network error. vSphere Replication reverts the replication image and reports a failed recovery task due to virtual machine registration error. If you initiate another recovery, it fails with a message that a virtual machine with the same name is already registered in vCenter Server.

  Workaround: Remove the partially recovered virtual machine from the vCenter Server inventory. Do not delete the files from disk. Try the recovery again.

• vSphere Replication operations fail when there is heavy replication traffic

  vSphere Replication operations might fail with error java.net.UnknownHostException. These errors occur because DNS requests are dropped due to network congestion.

  Workaround: Configure your network to ensure that management traffic is not dropped, by configuring traffic shaping, quality of service, or DNS on the vSphere Replication appliance. One possible solution is to modify the network address caching policy for the vSphere Replication appliance.

  1. Log into the vSphere Replication appliance as root.
  2. Open the file /usr/java/jre1.7.0_72/lib/security/java.security in an editor.
  3. Uncomment the line networkaddress.cache.ttl and set its value to at least 86400 seconds (24 hours) or to the longest time that is required for an initial full sync to complete.
  4. Save the file and reboot the vSphere Replication appliance.
  5. Repeat the procedure for all remaining vSphere Replication appliances.

Replications to vCenter Server

• Cannot reconnect sites if the connection status is "Connection issue"

  When two sites are connected, but the status of the connection is Connection issue, the following error message appears if you try to reconnect the sites:
  An internal error has occurred - scheme and schemeSpecificPart should not be null.

  Workaround: Disconnect the sites and connect them again.

• You cannot use custom defined users and roles with vSphere Replication

  You are unable to configure a replication with a custom user, even if that custom user is assigned all required VRM privileges on both sites. The error message Permission to perform this operation is denied appears on the Target Location page in Configure Replication wizards.

  Workaround: None. All vSphere Replication operations must be performed with the SSO administrator user on both sites.

• A recovered virtual machine with multiple point-in-time instances enabled can lose the attached disks to the latest snapshot when you revert to a previous snapshot and then revert to latest snapshot again

  When you recover a virtual machine for which you enabled point-in-time instances and attach a disk for unresolved disks, if any, the disks attach to the latest snapshot. If you revert to a previous snapshot and then revert to the latest one, the attached disks are not available.

  Workaround: Edit settings of the virtual machine and add the required disks as existing hard disks.

• When a target vSphere Replication server is not available, vSphere Replication does not show an error in the vSphere Web Client Monitor -> Issues page

  If the target vSphere Replication server is not available because it is powered off or has network connectivity issues, and a replication is in an initial full-sync state, vSphere Replication does not report an issue in the Web Client at Monitor -> Issues page of the target vCenter Server. Instead, you see an event on the vCenter Server and a disconnected status at Manage -> vSphere Replication -> Replication servers.

  Workaround: Check if a target vSphere Replication server is currently available at Manage -> vSphere Replication -> Replication Servers page. Alternatively, set an alarm for "VR Server disconnected" event on the target vCenter Server.

• Cannot reconfigure replication after switching from embedded database to existing external database

  If you configure vSphere Replication with an external database and configure replication within the same site, then switch to the embedded database, the replication is not available, which is as designed. If you switch back to the external database, the replication is in an error state. Reconfiguring the replication fails with the following error: ManagedObjectNotFound

  Workaround: When restoring the vSphere Replication database to the previous external or embedded database, you must reset its contents.

• Cannot configure a virtual machine with physical mode RDM disk even if the disk is excluded from replication

  If you configure a replication for a virtual machine with physical mode, you might see the following error:

  VRM Server generic error. Check the documentation for any troubleshooting information. 
  The detailed exception is: HMS can not set disk UUID for disks of VM : MoRef: 
  type = VirtualMachine, value = 
     
        
         
      , serverGuid = null'.
     
        

  Workaround: None.

• Recovering a virtual machine using the "Recover with latest available data" option is possible when the source virtual machine is powered on

  Before you start a recovery operation on the target site, you must power off the replication source virtual machine. However, if you select the option Recover with latest available data when recovering a virtual machine, it is possible to perform the recovery while the source virtual machine is powered on. This causes the following problem.

  • The network cards of the recovered virtual machine are disconnected when it powers on.

    Workaround: Ensure that the source virtual machine is powered off before you connect the recovered virtual machine to the network.

  If you select Recover with recent changes when you recover a virtual machine, it is not possible to complete the recovery if the source virtual machine is powered on.

• Recovering a virtual machine with vSphere Replication 6.5 fails to power on the recovered virtual machine

  If a replicated virtual machine is attached to a distributed virtual switch and you attempt to perform a recovery in an automated DRS cluster, the recovery operation succeeds but the resulting virtual machine cannot be powered on.

  Workaround: Edit the recovered virtual machine settings to attach it to the correct network.

• Registering additional vSphere Replication servers takes a long time

  If vCenter Server manages several hundred ESXi Server hosts, registering an additional vSphere Replication server with the vSphere Replication appliance can take several minutes.
  This is because the vSphere Replication server must register with each ESXi Server host.

Cloud Replications

• org.hibernate.exception in the VCTA log file

  In the vcta-info.log.<n> file or the vcta-debug.log file, you might observe the following message:
  org.hibernate.exception.ConstraintViolationException: Could not execute JDBC batch update

  Workaround: You can ignore this message because it does not affect the operation of vCloud Air - Disaster Recovery.

• The vApp in the cloud organization is not powered off after a recovery on premise

  When you recover a replication from cloud at the tenant site and, in the Recovery wizard, you select to recover the VM by using the option Use latest available data, vSphere Replication does not power off the source vApp in the cloud.
  This is because the option Use latest available data assumes that there is no connection to the replication source site.

  Workaround: You can connect to the cloud site to manually power off the source vApp.

• Replications from cloud turn into Error state

  If you use the vCloud Air web user interface to add a new disk to a virtual machine that serves as a replication source, vSphere Replication at your local site automatically pauses the incoming replication for that machine, and moves the replication group into Error state.

  Workaround: Stop the replication from cloud that indicates Error state, and configure a new replication.

• Hardware changes on the replication source VM might not be automatically copied to the placeholder vApp in the cloud

  Changes to the protected virtual machine on the source site, such as changes to memory, CPU, networks, and so on, might not be replicated to the placeholder vApp in your cloud organization if you apply them while vSphere Replication is running a workflow, for example, a test recovery.

  Workaround: Edit the hardware of the replication source VM again to trigger a full synchronization.

  1. In the vSphere Web Client inventory tree, right-click the source VM.
  2. From the drop-down menu, select Edit Settings, and apply a change to the virtual hardware.
     Note: Opening and closing the Edit Setting dialog box is not enough. You must apply some change to the hardware.
  3. Click OK.
• Disks are not automatically consolidated during recovery at the cloud site

  If you configure a replication to cloud that has the MPIT functionality enabled, and you recover the replicated virtual machine at the cloud site, its retained instances are not consolidated during the recovery. By design, replication instances are not consolidated to speed up the recovery process.
  The unconsolidated disks in the recovered virtual machine might cause performance problems as follows.

  • The recovered virtual machine runs slower than expected.
  • The recovered virtual machine requires more storage resources.

  Workaround: Use the vCloud Air interface to manually consolidate the disks on the recovered virtual machine.

  
  
  
• Outgoing replications to cloud remain in Not Active state

  By default, when you power on the vSphere Replication appliance, a vSphere Installation Bundle (VIB) is installed on all supported ESXi hosts in the vCenter Server inventory where the appliance is deployed. The VIB creates a firewall rule, Replication-to-Cloud Traffic, that opens TCP ports 10000 to 10010 for outgoing traffic. However, the automatic installation of the VIB file might fail due to network issues in your environment. When the firewall rule is missing on the source ESXi hosts, outgoing replications to cloud remain in Not Active state.
  

  Workaround: Install the vSphere Replication VIB file on each ESXi instance that hosts a cloud replication source VM.

  1. Temporarily disable the firewall on the ESXi host.
  2. Establish an SSH connection to the ESXi Server.
  3. Run the following command:
     $ esxcli software vib install -v https://VR_APPLIANCE_IP:8043/vib/vr2c-firewall.vib
  4. Enable the firewall on the ESXi host.
• Planned migration or synchronization fails with error: A replication error occurred at the vSphere Replication Server.

  If, during planned migration, the infrastructure (hosts, network, or storage) is under heavy load, running a planned migration might fail with one of the following errors.

  • A replication error occurred at the vSphere Replication Server for replication <group_name>. Details: 'Error for (datastoreUUID: "..."), (diskId: "..."), (hostId: "..."), (pathname: "..."), (flags: retriable): Class: NFC Code: 10; NFC error: The operation completed successfully; Set error flag: retriable; ...'
  • A replication error occurred at the vSphere Replication Server for replication <group_name>. Details: 'Error for (datastoreUUID: "..."), (diskId: "..."), (hostId: "..."), (pathname: "..."), (flags: retriable, pick-new-host, nfc-no-memory): Class: NFC Code: 5; NFC error: NFC_NO_MEMORY; Set error flag: nfc-no-memory; Code set to: Host unable to process request.; Set error flag: retriable; Set error flag: pick-new-host; Can't write (single) to remote disk'.
  
  Usually, these errors are transient and the operation succeeds if you retry running it.

  Workaround: If the error occurs frequently in your environment, you can increase the toleration period for replication synchronizations on the vSphere Replication Management Server (VRMS).

  1. Log in to the VRMS appliance as the root user and navigate to /opt/vmware/hms/conf/.
  2. Open the hms-configuration.xml file for editing and set the value of the hms-sync-replication-error-toleration-period property to 300000.
  3. Try running the planned migration task again.

• All operations on a seed vApp in vCloud Air are disabled

  If you configure a replication to cloud and select a vApp from the vCloud Air inventory to be used as a replication seed, all operations on the seed vApp are disabled.

  Workaround: None. Replication seeds cannot operate as virtual machines. A seed vApp can be used for only one replication.