As of vSphere 8.0 U3, VADP partners can repurpose existing applications to support backup and restore of virtual machines managed by IaaS Platform, formerly Tanzu. IaaS Platform includes two related components: VM Service for the vSphere side and VM Operator for the Kubernetes side.
Prerequisites: VADP partner applications must support first class disk (FCD), often used for Kubernetes storage, and container storage snapshots. FCD support goes back to vSphere 6.7.
VM Service in vSphere enables DevOps engineers to deploy and run containers, in addition to VMs, in a common Kubernetes environment. VMs and containers share the same Namespace resources and are managed with a single interface. VM Service helps developers use Kubernetes alongside non-containerized VM workloads, while allowing vSphere administrators to control service availability and resource consumption. VM Service manages virtual machines in coordination with VM Operator service.
A VM managed by VM Service is much like a regular VM, but includes VM custom resources, bootstrap Secret, and optionally FCD for persistent volume (PV) claims. It runs with Supervisor, which is a platform for natively hosting Kubernetes workloads. Sections below describe how vSphere administrators can perform backup and restore of VM Service managed VMs.
Backup: For a vendor of backup software, a VM managed by VM Service is similar to a regular VM, but also includes custom resources (and so forth) in the ExtraConfig section of its VMX file. Backups take place as usual, with administrators setting up an automated schedule for full and incremental backups. When triggered, backup software saves the VM configuration, disk data, and (for VM Service VMs) custom resources, bootstrap Secret, and storage claims.
VM Operator persists virtual machine custom resources (CR), in other words Kubernetes state, in the ExtraConfig section of the virtual machine's VMX file. Custom resources include information about paths and types of disks so that disks registered as FCD will be recognized as persistent volumes (PV) during restore operations. This ExtraConfig data is base64 encoded and gzip compressed to conserve space. Only custom resources owned by the DevOps persona are backed up. The following files are involved in ExtraConfig:
- vmservice.virtualmachine.resource.yaml – Kubernetes state and VM configuration.
- vmservice.virtualmachine.additional.resources.yaml – Additional Kubernetes resources needed to re-create the VM, such as a PV claims for volume resources, and the Secret resource required to bootstrap the machine.
- vmservice.virtualmachine.pvc.disk.data – Information necessary to register restored disks as PV claims when restoring with the VADP partner application.
As you can see in steps 1 and 2 of Back up VM Service VM
, normal backup operations are unchanged. A VM Service managed VM is exactly like any other VM that is available as a custom resource on Supervisor. This type of VM can also have additional FCD attached to it, expressed as PV claims on Supervisor. Third party software backs up a VM Service VM like any other VM by copying the VMX file and all attached disks along with other necessary files.
Restore: For a vendor of backup and restore software, restore operations are the same as for a regular VM. For vSphere administrators, restoring a VM Service VM is slightly different. They must be a member of the Administrators group. Upon restore, administrators must choose the Supervisor namespace resource pool where the VM gets restored, and specify a new name, or the same name, for the VM. After restore, automatic registration is triggered, reading ExtraConfig previously stored by backup, and restoring state on Supervisor. Registration is task based, so administrators can view progress and results in the vSphere Client. If errors occur, administrators can run registerVM as a REST command.
When required, the virtual infrastructure (VI) administrator runs their backup vendor's application to restore the VM in the resource pool corresponding with the Supervisor namespace. The VM should be restored in powered-off state. Ideally, workload control plane (WCP) service detects when a VM is restored in a Supervisor namespace resource pool by a backup application and calls registerVM. As a result, a VirtualMachine custom resource is created on Supervisor along with other necessary resources. If a bootstrap resource was specified when creating the VirtualMachine custom resource, the restore process also creates those resources.
VADP based restore is followed by automatic registration of the restored VM as a VirtualMachine custom resource on Supervisor. Automatic registration is tried only once so as to allow vSphere administrators to correct registration issues in case of failure. If automatic registration fails, the administrator can run the registerVM vAPI manually after resolving any issues. One such issue might be Supervisor namespace lacking access to the storage policy being referenced by the VirtualMachine resource.
A common workflow is to restore the VM with a different name, such as with added suffix or prefix. The name of the restored VM takes precedence over the name recorded in ExtraConfig during backup.
After naming, the VM Operator applies webhooks to check that the created VirtualMachine resource is a valid configuration. Webhooks checking might fail if the underlying infrastructure changed between backup and restore, for instance if a StorageClass associated namespace was removed. To troubleshoot such cases, vAPI returns the exact error received from the server.
As you can see in steps 1 and 2 of Restore VM Service VM
, operations are similar to normal restore, except the VM is managed by VM Service, and its VMDK could be FCD. WCP watches for a newly restored VM and attempts to auto-register it. If all goes well, the administrator does not need to intervene again.
IP address management of the restored VM is handled as for a new VM, by the VM operator making connection requests to the network provider.
