Understanding VM Sizing, VM Placement, and vGPU Policies in VMware Cloud Director

In VMware Cloud Director Service Provider Admin Portal, you can control the virtual machine (VM) resource allocation and placement on a specific cluster or host by using VM sizing policies, VM placement policies, and vGPU policies.

VMware Cloud Director system administrators create and manage VM sizing, VM placement, and vGPU policies at a global level and can publish individual policies to one or more organization virtual data centers (VDCs). Additionally, if a user saves a vApp as a vApp template to a catalog, the template also includes the vGPU, placement, and sizing policies of the original vApp as unmodifiable tagged policies.

When you publish a policy to an organization VDC, the policy becomes available to the users in the organization. When creating and managing VMs in the organization VDC, tenants can assign the available policies to the VMs. Tenants and users in the organization VDC cannot look into the specific configuration of a policy.

VM placement, VM sizing, and vGPU policies are a mechanism for cloud providers to define and offer differentiated levels of service, for example, a CPU intensive profile or a high memory usage profile. If you publish multiple VM placement, VM sizing, and vGPU policies to an organization VDC, tenant users can select between all custom policies and the default policy when creating and managing VMs in the organization VDC. The System Default policy is auto-generated for every VDC. You can delete the System Default policy in the VDC and mark another custom policy as the default. Every VDC must have a default compute policy. You can set as a default policy of a VDC any of the three compute policy types. The automatically generated System Default policy does not define any values and allows all virtual machine configurations.

VM placement policy

A VM placement policy defines the placement of a virtual machine on a host or group of hosts. It is a mechanism for cloud provider administrators to create a named group of hosts within a provider VDC. The named group of hosts is a subset of hosts within the provider VDC clusters that might be selected based on any criteria such as performance tiers or licensing. You can expand the scope of a VM placement policy to more than one provider VDC.

A VM placement policy defines VM-host affinity rules that directly impact the placement of tenant workloads. Administrators define or expose named host groups by using VM groups in vCenter Server. A VM group has a direct affinity to a host group and represents the host group to which it has the affinity.

You define the VM placement policy at the provider VDC level. A VM placement policy includes the following attributes:

Name (must be unique in the provider VDC)
Description
A set of one or more VM groups selected from the underlying clusters in the provider VDC. You can select one VM group per cluster

A VM placement policy is optional during the creation of a virtual machine and a tenant can assign only one VM placement policy to a virtual machine.

When a tenant creates a virtual machine in the organization VDC and selects the VM placement policy, VMware Cloud Director adds the virtual machine to the VM group or VM groups that are referenced in the policy. As a result, VMware Cloud Director creates the virtual machine on the appropriate host.

A VM placement policy can have zero or one VM group from each cluster. For example, the VM placement policy oracle_license can comprise VM groups oracle_license1 and oracle_license2, where VM group oracle_license1 belongs to cluster oracle_cluster1, and VM group oracle_license2 belongs to cluster oracle_cluster2.

When you assign a VM placement policy to a virtual machine, the placement engine adds this virtual machine to the corresponding VM group of the cluster on which it resides. For example, if you select to deploy a virtual machine on cluster oracle_cluster1 and assign the VM placement policy oracle_license to this virtual machine, the placement engine adds the virtual machine to VM group oracle_license1.

VM sizing policy

A VM sizing policy defines the compute resource allocation for virtual machines within an organization VDC. The compute resource allocation includes CPU and memory allocation, reservations, limits, and shares.

With VM sizing policies, VMware Cloud Director system administrators can control the following aspects of compute resources consumption at the virtual machine level:

Number of vCPUs and vCPU clock speed
Amount of memory allocated to the virtual machine
Memory and CPU reservation, limit, and shares
Extra Configurations.
The extraConfigs API parameter represents a mapping between a key and value pairs that are applied as extra configuration values on a virtual machine. You can create a policy with extra configurations only by using the vCloud API. Existing extra configurations appear in the Service Provider Admin Portal UI under Extra Configurations in the detailed VM sizing policy view.

You define the VM sizing policies at a global level. For more information about the VM sizing policy attributes, see Attributes of VM Sizing Policies in VMware Cloud Director.

VMware Cloud Director generates a default VM sizing policy for all VDCs. The default VM sizing policy contains only a name and description, and all remaining policy attributes are empty.

You can also define another VM sizing policy as the default policy for an organization VDC. The default VM sizing policy controls the resource allocation and consumption of the virtual machines that tenants create in the organization VDC, unless a tenant assigns another specific VM sizing policy to the virtual machine.

To limit the maximum compute resources that tenants can allocate to individual virtual machines within an organization VDC, cloud providers can define a maximum VM sizing policy. When assigned to an organization VDC, the maximum VM sizing policy acts as an upper bound for the compute resource configuration for all virtual machines within the organization VDC. The maximum VM sizing policy is not available to tenant users when creating a virtual machine. When you define a VM sizing policy as the maximum policy, VMware Cloud Director copies internally the content of the policy and uses the copied content as the maximum VM sizing policy. As a result, the organization VDC does not depend on the initially used VM sizing policy.

By using VM sizing policies, cloud providers can restrict the compute resources consumption for all virtual machines within an organization VDC to, for example, three predefined sizes, Small Size, Medium Size, and Large Size. The workflow is the following.

A system administrator creates three VM sizing policies with the following attributes.


Name	Attributes
Small Size	Description: Small-sized VM policy Name: Small Size Memory: 1024 Number of vCPUs: 1
Medium Size	Description: Medium-sized VM policy Name: Medium Size Memory: 2048 Number of vCPUs: 2
Large Size	Description: Large-sized VM policy Name: Large Size Memory: 4096 Number of vCPUs: 4

Publish the new VM sizing policies to an organization VDC.
Optionally define one of the VM sizing policies as a default VM sizing policy for the organization VDC.

vGPU policy

Starting with VMware Cloud Director 10.3.2, you can create, manage, and publish vGPU policies. When creating a VM, tenant users can select between creating a VM for general purposes or a vGPU enabled VM that requires vGPU resources.

The vGPU policy defines the number of GPU PCI devices that a VM must have. A vGPU policy must have a vGPU profile and its count information. Optionally, you can add placement and sizing information to a vGPU policy.

A vGPU policy can define the placement of a VM on a host or group of hosts. It is a mechanism for cloud provider administrators to create a named group of hosts within a provider VDC. The named group of hosts is a subset of hosts within the provider VDC clusters that might be selected based on any criteria such as performance tiers or licensing. You can expand the scope of a vGPU policy to more than one provider VDC.

A vGPU policy defines VM-host affinity rules that directly impact the placement of tenant workloads. Administrators define or expose named host groups by using VM groups in vCenter Server. A VM group has a direct affinity to a host group and represents the host group to which it has the affinity.

You define the vGPU policy at the global or provider VDC level. A vGPU policy includes the following attributes:

Name (unique globally)
Description
vGPU profile and its count
VMware Cloud Director loads the available vGPU profiles from vCenter Server. When the PCI count is 1, a VM gets 1 vGPU PCI device, when the count is 2, a VM gets 2 vGPU PCI devices, and so on.
Provider VDC scope
A vGPU policy can define its scope to all or a subset of provider VDCs in the system. For each provider VDC in the scope, the policy can further define the scope to one or more participating clusters of that provider VDC.
Host level placement
A set of one or more VM groups selected from the underlying clusters in the provider VDC. You can select one VM group per cluster
Sizing information
You can define the CPU and memory settings as part of a vGPU policy or during the creation of a VM, you can select a different available sizing policy.

A vGPU policy is mandatory during the creation of a vGPU enabled VM, and a tenant can assign only one vGPU policy to a VM.

When a tenant creates a VM in the organization VDC and selects the vGPU policy, VMware Cloud Director configures the VM to have vGPU PCI devices matching the vGPU profile and its count in the vGPU policy. VMware Cloud Director also adds the VM to the VM group or VM groups that are referenced in the policy. As a result, VMware Cloud Director creates the VM on the appropriate host.

A vGPU policy can have zero or one VM group from each cluster. For example, the vGPU policy oracle_license can comprise VM groups oracle_license1 and oracle_license2, where VM group oracle_license1 belongs to cluster oracle_cluster1, and VM group oracle_license2 belongs to cluster oracle_cluster2.

When you assign a vGPU policy to a VM, the placement engine adds this VM to the corresponding VM group of the cluster on which it resides. For example, if you select to deploy a VM on cluster oracle_cluster1 and assign the vGPU policy oracle_license to this VM, the placement engine adds the VM to VM group oracle_license1.

The available policy operations for cloud providers are the following:

To define the placement of a VM on a host or group of hosts, create a placement policy. See Create a VM Placement Policy within a Provider VDC in VMware Cloud Director.
To control the physical compute resource allocation for tenant workloads, create a sizing policy. See Create a VM Sizing Policy in VMware Cloud Director.
Тo define the placement and compute resource allocation of a vGPU enabled VM, create a vGPU policy. See Create a vGPU Policy in VMware Cloud Director.
Publish a policy to one or more organization VDCs. See Add a VM Placement Policy to an Organization VDC in VMware Cloud Director, Add a VM Sizing Policy to an Organization VDC in VMware Cloud Director, and Add a vGPU Policy to an Organization VDC in VMware Cloud Director.
Set a VM placement policy, a VM sizing policy, or a vGPU policy as default.
Edit a policy.
Unpublish a policy from an organization VDC.
Delete a policy. See Delete a VM Placement Policy From VMware Cloud Director, Delete a VM Sizing Policy From VMware Cloud Director, and Delete a vGPU Policy from VMware Cloud Director.

Users that have the ORG_VDC_MANAGE_COMPUTE_POLICIES right can create, update, and publish VM sizing, VM placement, and vGPU policies.

The following table lists the available VM sizing policy, VM placement, and vGPU policy operations for tenant users.

Table 1. VM Sizing Policy, VM Placement, vGPU Policy Operations for Tenant Users
Operation	Description
Assign a policy to a virtual machine during a virtual machine creation.	Tenant users that are authorized to create virtual machines in an organization VDC can optionally assign VM sizing, VM placement, and vGPU policies to VMs by using the VMware Cloud Director Tenant Portal. As a result, the parameters defined in a VM sizing policy control the CPU and memory consumption of the VM. The sizing parameters defined in a vGPU policy can also optionally control the CPU and memory consumption of the VM. Assigning a VM placement or sizing policy is not a requirement for tenants during a VM creation. If a tenant does not explicitly select a VM sizing policy to assign to a VM, the default VM sizing is applied to the virtual machine. If you do not create and publish any VM placement policies, the VM placement policy option is not visible to the tenants. If you do not create and publish any vGPU policies, the tenants can create only general purpose VMs. If a tenant selects a VM placement or vGPU policy that has sizing information, the VM sizing policy option becomes hidden to the tenant. You can create a VM placement policy with sizing information only by using the vCloud API. If there is only one VM sizing policy, the VM sizing policy option is not visible to the tenants. When the system administrator sets the vCPU Count, Cores Per Socket, and Memory attributes in a VM sizing or vGPU policy, if a tenant selects the policy, these values are shown, but not editable.
Assign a policy to an existing virtual machine.	Tenant users that are authorized to manage VMs in an organization VDC can assign or change the VM sizing, VM placement, and vGPU policies of an existing VM using the VMware Cloud Director Tenant Portal. When a tenant changes the VM placement or vGPU policy, the VM moves to a new host as per the VM-host affinity rule defined in the new policy. When a tenant changes a VM sizing policy or vGPU policy with defined sizing information, the system reconfigures the VM to consume compute resources as specified in the new policy.

The workflow for working with VM placement and VM sizing policies is the following.

You creates one or more VM placement policies. See Create a VM Placement Policy within a Provider VDC in VMware Cloud Director.
When you create a VM placement policy to be scoped for a single provider VDC, the name of the VM placement policy must be unique within the provider VDC scope of the policy. If you create a VM placement policy scoping more than one provider VDC, the VM placement policy name becomes unique globally within the VMware Cloud Director site.
You create one or more VM sizing policies. See Create a VM Sizing Policy in VMware Cloud Director.
The names of VM sizing policies are unique in a single VMware Cloud Director site.
You create one or more vGPU policies. See Create a vGPU Policy in VMware Cloud Director.
The names of vGPU policies are unique in a single VMware Cloud Director site.
You publish the VM placement, VM sizing, and vGPU policies to one or more organization VDCs. See Add a VM Placement Policy to an Organization VDC in VMware Cloud Director, Add a VM Sizing Policy to an Organization VDC in VMware Cloud Director, and Add a vGPU Policy to an Organization VDC in VMware Cloud Director.
Publishing a policy makes it available for tenant users in the organization VDCs during virtual machine creation and virtual machine editing.
When creating or updating a VM, tenants can use the VMware Cloud Director API or the VMware Cloud Director Tenant Portal to assign a VM sizing, VM placement, and vGPU policy to a VM.