VMware Integrated OpenStack 5.1.0.2 Release Notes

Updated on 4 JUN 2019

VMware Integrated OpenStack 5.1.0.2 | 23 MAY 2019 | Build 13714880

Check for additions and updates to these release notes.

What's in the Release Notes

The release notes cover the following topics:

About VMware Integrated OpenStack
Compatibility
Upgrading to Version 5.1.0.2
Deprecation Notices
Internationalization
Open Source Components for VMware Integrated OpenStack
Resolved Issues
Known Issues

About VMware Integrated OpenStack

VMware Integrated OpenStack greatly simplifies deploying an OpenStack cloud infrastructure by streamlining the integration process. VMware Integrated OpenStack delivers out-of-the-box OpenStack functionality and an easy configuration workflow through a deployment manager vApp that runs directly in vCenter Server.

Compatibility

See the VMware Product Interoperability Matrices for details about the compatibility of VMware Integrated OpenStack with other VMware products, including vSphere components.

Upgrading to Version 5.1.0.2

The upgrade to VMware Integrated OpenStack 5.1.0.2 is a patch process. If you are running VMware Integrated OpenStack 5.1 or 5.1.0.1, see the Patch Process section. If you are running VMware Integrated OpenStack 5.0 or an earlier version, see the Earlier Versions section.

Patch Process

If you are running VMware Integrated OpenStack 5.1 or 5.1.0.1, you can apply the patch directly to your existing deployment. To do so, perform the following steps:

Verify that the VMware Integrated OpenStack service is either running or not yet deployed. If the VMware Integrated OpenStack service is in any other state, the upgrade will fail.
In the vSphere Client, take a snapshot of the OpenStack Management Server virtual machine.
On the OpenStack Management Server virtual machine, take a snapshot by running the following command:
sudo viopatch snapshot take
This command stops OpenStack services. Services will be started again when the patch is installed.
NOTE: If the command fails, see "For deployments using a remote vCenter Server, the viopatch command fails to take snapshots" in the Known Issues section.
Download the patch file to the OpenStack Management Server virtual machine.
Add the patch file by running the following command:
sudo viopatch add -l path/vio-patch-5.1.0.2_5.1.0.13714880_all.deb
Install the patch file by running the following command:
sudo viopatch install -p vio-patch-5.1.0.2 -v 5.1.0.13714880
During the patch installation, the API endpoint is automatically turned off. As a result, any API calls during installation will return a 503 error.
Log out of the vSphere Client and log back in. Any error messages encountered during login can be ignored.

NOTE: The viopatch uninstall action is deprecated and cannot be used to revert to the previous version. The snapshots created in this process are therefore necessary for reversion. Do not remove these snapshots until all validation tasks have been completed and you are certain that you will not need to revert to the previous version.

After you have validated that the patched version is operating correctly, you can run sudo viopatch snapshot remove to delete the snapshot. This action is destructive and cannot be reversed.

If you need to revert to the previous version after installing the patch, perform the following steps.

In the vSphere Client, revert the OpenStack Management Server virtual machine to the previous snapshot.
On the OpenStack Management Server virtual machine, restart the OpenStack service by running the following command:
sudo service oms restart
Revert to the previous snapshot by running the following command:
sudo viopatch snapshot revert
On the vCenter Server virtual machine, stop the vSphere Client service, delete residual files, and restart the service:
service-control --stop vsphere-ui cd /etc/vmware/vsphere-ui/vc-packages/vsphere-client-serenity/ rm -rf * cd /usr/lib/vmware-vsphere-client/server/work rm -rf * service-control --start vsphere-ui
Log out of the vSphere Client and log back in.

Earlier Versions

To upgrade from VMware Integrated OpenStack 5.0 to VMware Integrated OpenStack 5.1.0.2, perform the following steps:

Upgrade to VMware Integrated OpenStack 5.1 as described in Patch VMware Integrated OpenStack.
Apply the VMware Integrated OpenStack 5.1.0.2 patch as described above in the Patch Process section.

To upgrade from VMware Integrated OpenStack 4.x to VMware Integrated OpenStack 5.1.0.2, perform the following steps:

Deploy the VMware Integrated OpenStack 5.1 OVA as described in Install the New Version.
On the new 5.1 deployment, apply the VMware Integrated OpenStack 5.1.0.2 patch as described above in the Patch Process section.
Migrate to the patched deployment as described in Migrate to the New VMware Integrated OpenStack Deployment.

Deprecation Notices

New versions of vRealize Automation will no longer be certified with VMware Integrated OpenStack.

Some OpenStack Management Server lifecycle management APIs available in VMware Integrated OpenStack 5.1.0.2 will be changed or deprecated in a future major release of VMware Integrated OpenStack.

The SDDC provider in VMware Integrated OpenStack with Kubernetes, intended for deployments without VMware Integrated OpenStack, will be deprecated in a future major release. For new VMware Integrated OpenStack with Kubernetes deployments, use the OpenStack provider.

Internationalization

VMware Integrated OpenStack 5.1.0.2 is available in English and seven additional languages: Simplified Chinese, Traditional Chinese, Japanese, Korean, French, German, and Spanish.

The following items must contain only ASCII characters:

Names of OpenStack resources (such as projects, users, and images)
Names of infrastructure components (such as ESXi hosts, port groups, data centers, and datastores)
LDAP and Active Directory attributes

Open Source Components for VMware Integrated OpenStack

The copyright statements and licenses applicable to the open source software components distributed in VMware Integrated OpenStack 5.1.0.2 are available on the Open Source tab of the product download page. You can also download the disclosure packages for the components of VMware Integrated OpenStack that are governed by the GPL, LGPL, or other similar licenses that require the source code or modifications to source code to be made available.

Resolved Issues

For NSX Data Center for vSphere deployments, SpoofGuard limitations prevent the association of certain types of IP addresses with a virtual machine.
In earlier versions, it was not possible to configure virtual machines to have port security and also be associated with a subnet or an IP address already used on the network.

It is now possible to relax port security requirements by disabling SpoofGuard, which allows the use of distributed firewall features from OpenStack on those virtual machines.

To enable this feature, perform the following steps:
1. On the OpenStack Management Server, open the custom.yml file and set the value of nsxv_allow_multiple_ip_addresses to true.
2. Run the sudo viocli deployment configure command.
3. Ensure that port security is disabled on the target network.
4. Ensure that port security is enabled on the target port.
You can then use the --allowed-address-pairs parameter to specify a single IP address or entire subnet when creating or updating a port. Because ports inherit the port security setting from the network at the time of port creation, you must manually enable port security on ports that you create after disabling port security on the network. Disabling or enabling port security on the network does not affect the port security setting of existing ports.

Note: SpoofGuard must be disabled on the network because its granularity is at the logical switch level.
Additional parameters have been added to the custom.yml file.
The following items can now be configured in custom.yml:
- nova_default_vif_model - sets the default vNIC model.
- nova_additional_passthrough_devices - includes additional passthrough devices (as a series of integers).
- glance_api_image_member_quota - sets the maximum number of users with whom an image can be shared.
Instances cannot be launched from images using thick provisioning.
When you use the vmware_template_disk_type metadata to set the provisioning type of an image to thick, instances fail to launch from that image.

This issue has been resolved in this release.
The VMware Integrated OpenStack dashboard does not allow you to select HTTPS for LBaaS health monitors.
LBaaS health monitors can be configured with HTTPS over the command-line interface, but not the dashboard.

This issue has been resolved in this release.
Attempting to modify an image without sufficient permissions causes the user to be logged out.
If you are logged in as a user with the _member_ role and attempt to modify an image owned by a cloud administrator, you will be automatically logged out of the VMware Integrated OpenStack dashboard.

This issue has been resolved in this release.
VMware Integrated OpenStack cannot send logs to vRealize Log Insight over the TLS protocol.
In earlier versions, it was not possible to use TLS encryption on communications with vRealize Log Insight.

This issue has been resolved in this release.
The server rescue feature does not function properly.
When you use the openstack server rescue command to reboot an instance into rescue mode, an error occurs and the operation fails.

This issue has been resolved in this release. However, the following limitation applies:

If you want to use the rescue feature, you must set the value of vmware_create_template to false when uploading the image for your instance. For example:
```
openstack image create test-image --disk-format vmdk --container-format bare --file test-image.vmdk --property vmware_disktype="preallocated" --property vmware_create_template="false"
```
If you change the disk adapter type of an image, instances controlled by a storage policy fail to launch from that image.
If you use the vmware_adaptertype metadata on an image (for example, to change its adapter from IDE to SCSI), instances that are under Storage Policy-Based Management (SPBM) can no longer be launched using that image.

This issue has been resolved in this release.
Virtual machines cannot be imported into VMware Integrated OpenStack.
When you attempt to import an unmanaged virtual machine, a method invocation error occurs.

This issue has been resolved in this release.
Serial console logs are not rotated and may occupy excessive disk space.
Log files generated by the Virtual Serial Port Concentrator (vSPC), located in the /var/log/vspc directory on the compute node, are not rotated and can eventually take up too much disk space.

This issue has been resolved in this release.
JSON template files exported from the Flex and HTML5 clients are not completely interchangeable.
If you export a template using the Flex-based vSphere Web Client and import it using the HTML5 vSphere Client, the "Create Metadata Proxy Server" and "Create DHCP Server Profile" settings are not preserved.

This issue has been resolved in this release.
The viocli upgrade mgmt_server command may fail to complete.
Due to an upstream OpenStack issue, online data migration may fail on some deployments. The viocli upgrade mgmt_server command then hangs and the upgrade process cannot be completed.

This issue has been resolved in this release.
Snapshots cannot be taken of instances in a second availability zone.
When you create a snapshot of an instance that is not in the first availability zone, the snapshot process fails to complete.

This issue has been resolved in this release.
The edge cluster drop-down list is unavailable when deploying OpenStack from a JSON template.
In the Flex-based vSphere Web Client, if you deploy OpenStack using a template file with "Create Metadata Proxy Server" or "Create DHCP Server Profile" selected, you are unable to select an edge cluster from the drop-down list.

This issue has been resolved in this release.
The OpenStack command-line interface displays some user names as "None".
When you use the openstack user list command, some user names are returned as "None" even though a valid name has been assigned.

This issue has been resolved in this release.

Known Issues

The known issues are grouped as follows.

VMware Integrated OpenStack
VMware Integrated Openstack with Kubernetes

VMware Integrated OpenStack

In an environment with multiple vCenter Server instances, Nova live migration cannot move an instance to a host in the remote vCenter Server.
Nova live migration across vCenter Server instances is not supported.

Workaround: None.
Deleting an instance associated with a floating IP address does not delete associated DNS records.
If you delete an instance that has a floating IP address associated, Designate will not remove the DNS records for that instance.

Workaround: Disassociate the floating IP address from the instance before deleting the instance.
Compute nodes might not start after being removed and re-added.
If there is a tenant virtual data center created for a compute node, that node will fail to start if it is removed and added again.

Workaround: None.
Local storage may be incorrectly calculated on the VMware Integrated OpenStack dashboard.
If multiple compute nodes use the same datastore, the Hypervisors page on the VMware Integrated OpenStack dashboard will incorrectly display that the total disk space available is the size of the single datastore multiplied by the number of compute nodes using it. In addition, the entry in the Local Storage (used) column for each compute node will display the total used space on the datastore, not the used space for a single compute node.

Workaround: None.
In an environment with multiple vCenter Server instances, instances cannot find templates and fail to boot.
If an image is deleted manually from the compute vCenter Server, instances might fail to boot with the error "Unable to find template at location". This issue can also occur when re-adding compute nodes.

Workaround: Determine the location of the remote vCenter Server from the image by running the glance image-show image-uuid command. Then delete the location from Glance by running the glance location-delete --url image-location command.
For deployments using a remote vCenter Server, the viopatch command fails to take snapshots.
In a deployment where all control virtual machines are deployed in a management vCenter Server instance and use a Nova compute node deployed in a remote vCenter Server instance, the viopatch snapshot take command cannot obtain information about the management vCenter Server instance. The command fails with the error "AttributeError: 'NoneType' object has no attribute 'snapshot'."

Workaround: On the OpenStack Management Server virtual machine, manually set the IP address, username, and password of the management vCenter Server by running the following commands:
```
export VCENTER_HOSTNAME = mgmt-vc-ip-address
export VCENTER_USERNAME = mgmt-vc-username
export VCENTER_PASSWORD = mgmt-vc-password
```
VMware Integrated OpenStack cannot connect to NSX-T after the NSX-T password is changed.
If you change the NSX-T password while the Neutron server is running, VMware Integrated OpenStack might fail to connect to NSX-T.

Workaround: Before changing the NSX-T password, log in to the active controller node and run the systemctl stop neutron-server command to stop the Neutron server service. The service will be restarted after you update the NSX-T password in VMware Integrated OpenStack.
The Nova compute service fails to start after an upgrade from version 4.x.
If a Nova compute node was deleted in version 4.x and a new Nova compute node using the same vCenter Server and same cluster was added later, the Nova compute service will fail to start after you upgrade to version 5.x. "ERROR nova ResourceProviderCreationFailed" is written to /var/log/nova/nova-compute.log.

Workaround: Perform the following steps to remove the Nova compute node from the database:
1. Find the MOID of the deleted compute node.
2. Log in to the active database node and open the nova_api database:
  mysql use nova_api
3. In the "resource_providers" table, remove the resource_provider record with the MOID of the deleted compute node.
4. In the "host_mappings" table, remove the host record for the deleted compute node.
A datastore failure might render the OpenStack deployment inaccessible.
If all nodes in an HA deployment use the same datastore, the failure of that datastore will cause the entire deployment to be inaccessible.

Workaround: Try to fix the failed datastore and recover its data. After the virtual machine for each node is shown in vCenter Server, restart the OpenStack deployment. If the datastore is not recoverable, use the viocli recover command to restore the failed nodes.
The Keystone endpoint is in the error state.
After the internal endpoint in-flight encryption setting is changed, the Keystone endpoint fails to reconnect. This issue occurs when you set the internal_api_protocol parameter to http for an HA deployment or https for a compact or tiny deployment.

Workaround: Modify the Keystone endpoint URL.
1. In the vSphere Web Client, select Administration > OpenStack.
2. Select the KEYSTONE endpoint and click the Edit (pencil) icon.
3. In the Update Endpoint section displayed, change the URL to begin with http or https depending on your configuration.
4. Enter the administrator password and click Update.
The VMware Integrated OpenStack OVA cannot be deployed in the HTML5 vSphere Client in vCenter Server 6.7.
After you deploy the VMware Integrated OpenStack OVA using the HTML5 vSphere Client in vCenter Server 6.7, the VMware Integrated OpenStack vApp fails to power on and the UI displays the error: "The virtual machine has a required vService dependency 'vCenter Extension Installation' which is not bound to a provider."

Workaround: Deploy the VMware Integrated OpenStack OVA using the Flex-based vSphere Web Client or using the OVF Tool.

For more information, see the vSphere 6.7 Release Notes and KB 55027.
Host names that start with a number cause a "java.io.IOException" error.
The OpenStack Management Server does not support host names that start with a number. The error "java.io.IOException: DNSName components must begin with a letter" appears if the host name starts with a number.

Workaround: Use a host name that does not start with a number. For more information, see the JDK upstream issue: https://bugs.openjdk.java.net/browse/JDK-8054380
East-west traffic does not travel between virtual machines booted on a virtual wire provider network.
If you create a provider network using virtual wire and do not create a SpoofGuard policy, east-west traffic will not travel between the virtual machines booted on this provider network.

Workaround: Create a SpoofGuard policy and add virtual wire to the policy before creating the virtual wire provider network.
Certificate verification may fail on the OpenStack Management Server.
When you use the viocli command-line utility, the following error may occur:
```
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)
```
Workaround: On the OpenStack Management Server, disable verification of the vCenter Server certificate by running the following commands:
```
sudo su -
export VCENTER_INSECURE=True
```
Deleting a router interface times out.
When concurrent Heat stacks are deployed with shared NSX routers, router interface deletion can time out. The following might be displayed: neutron_client_socket_timeout, haproxy_neutron_client_timeout, or haproxy_neutron_server_timeout.

Workaround: Do not use shared routers in environments where network resources frequently change. If NAT or floating IP addresses are required, use an exclusive router. Otherwise, use a distributed router.

VMware Integrated Openstack with Kubernetes

For NSX-T deployments, the vkube cluster heal command might fail.
If you use the vkube cluster heal command on a cluster whose k8s-master-0 node is in the Error state, the following error message may be displayed:
```
fatal: [k8s-master-0-0ffeac43-78ea-4eab]: FAILED! => {"changed": false, "msg": "Unable to start service etcd: Job for etcd.service failed because a timeout was exceeded. See \"systemctl status etcd.service\" and \"journalctl -xe\" for details.\n"}
```
Workaround: Using SSH, log in to the k8s-master-1 node for the affected cluster and manually delete the unreachable member from the etcd cluster. Then close the SSH connection and run the vkube cluster heal command again.
A cluster in the ERROR state cannot be deleted.
If the infrastructure is out of resources, cluster creation, healing, and scaling will fail and the cluster will enter the ERROR state.

Workaround: Perform the following steps:
1. Log in to the toolbox container.
2. Use the OpenStack client to delete hosts in the ERROR state.
3. Run the vkube cluster delete command again to delete the cluster.
If a username or domain contains a backslash (\), authentication fails.
The Keystone authentication plugin uses the backslash character as a separator to encode the domain name and username in a single string . If there is an additional backslash in either the domain name or username, the Keystone authentication plugin will not decode the domain name and username correctly.

Workaround: Use domain names or usernames that do not include the backslash character.