Instance with NUMA Affinity

VMware Integrated OpenStack supports non-uniform memory access (NUMA)-aware placement of OpenStack instances on the underlying vSphere environment.

Important: This feature is offered in VMware Integrated OpenStack Carrier Edition only. For more information, see VMware Integrated OpenStack Licensing.

NUMA links small, cost-effective nodes using a high-performance connection to provide low latency and high throughput. This performance is often required for virtual network functions (VNFs) in telecommunications environments. For information about NUMA in vSphere, see Using NUMA Instances with ESXi in vSphere Resource Management.

To obtain information about your current NUMA configuration, run the following command on your ESXi hosts:

vsish -e get /net/pNics/vmnic<id>/properties | grep 'Device NUMA Node'

Prerequisites

Ensure that vCPUs, memory, and physical NICs intended for virtual machine traffic are placed on same node.
In vSphere, create a teaming policy that includes all physical NICs on the NUMA node. See Teaming and Failover Policy in vSphere Networking.

Procedure

Log in to the Integrated OpenStack Manager as the root user.
```
ssh root@mgmt-server-ip
```
Open the toolbox and set the password for the admin account.
```
toolbox
export OS_PASSWORD=admin-account-password
```
Create a Neutron network on which all physical NICs are located on a single NUMA node.

Create an OpenStack flavor that includes the numa.nodeAffinity property.

nova flavor-key flavor-id set vmware:extra_config='{"numa.nodeAffinity": "numa-node-id"}'

Launch an OpenStack instance using the flavor and network created in this procedure.

Instance with Multi SRIOV

You can configure VM vNIC with multi SRIOV to balance workload on multiple Physical Network Adapters (pnic) to avoid network traffic congestion on one pnic while no traffic on another pnic. This application can be NUMA aligned with pnic to avoid performance loss. Two pnic work as pairs for physical connectivity redundancy, and different pairs for differeny traffic or VLAN.

Note: In VMware Integrated OpenStack 7.2, if you want to specify nodeAffinity, you must set it in flavor. You cannot configure after instance is created. Also, if numa.nodeAffinity is already specified in flavor, it overrides the NUMA setting from pnic.

The following steps describe the procedure to create instance with multi SRIOV:

Configure nova-compute.conf.

For the PCI definition in the Nova Compute CR, you can use the following format to input the content in the values section:

'{"address": "<PCI address of vmnicX>", "product_id": "*", "vendor_id": "*", "physical_network":"<uuid of vlan transport zone of provider-networkA>"}'

viocli update novacompute <nova-compute-cr-name>
conf:
  nova_compute:
    DEFAULT:
     ......
    pci:
      passthrough_whitelist:
        type: multistring
        values:
        - '{"address": "0000:1a:00.0", "product_id": "*", "vendor_id": "*", "physical_network":"bf86b52f-a629-4c07-a8bd-14b4b46ba384"}'
        - '{"address": "0000:5e:00.1", "product_id": "*", "vendor_id": "*", "physical_network":"c12a7025-22bc-403c-8ff9-9a25a236704a"}'
        - '{"address": "0000:d8:00.0", "product_id": "*", "vendor_id": "*", "physical_network":"801cd687-b65b-449c-9892-c22647851bf3"}'
        - '{"address": "0000:d8:00.1", "product_id": "*", "vendor_id": "*", "physical_network":"7ab64053-9e48-416d-8b1e-80e1fccd3dcd"}'

Create provider network and subnet.

neutron --insecure net-create 3158-net --provider:network_type vlan --provider:physical_network c12a7025-22bc-403c-8ff9-9a25a236704a --provider:segmentation_id 3158
openstack --insecure subnet create --network 3158-net --allocation-pool start=172.17.2.1,end=172.17.2.254 --gateway 172.17.0.1 --subnet-range 172.17.0.0/22 3158-subnet

Create SRIOV ports.

neutron --insecure port-create 3158-net --name 3158-p1 --vnic_type direct
neutron --insecure port-create 3158-net --name 3158-p2 --vnic_type direct
neutron --insecure port-create 3158-net --name 3158-p3 --vnic_type direct

Launch an instance with the specified port. You can see the network adapter added to the virtual machine and it is connected to vmnic2

nova boot --flavor m1.tiny --image a133ffd7-1601-42e5-a66f-1eb6130db093 --nic port-id=031e14e3-cf06-40e1-addf-3f13f5b5da66 --nic port-id=25c6ce90-2cff-4189-ae21-abc27a131ae7  <vm-name>