Using VMware Tanzu Mission Control, you can protect the valuable data resources in your Kubernetes clusters using the backup and restore functionality provided by Velero, an open source community standard.
- all resources in a cluster
- selected or excluded namespaces in a cluster
- specific or excluded resources in a cluster identified by a given label
- the entire backup
- selected or excluded namespaces from the backup
- specific or excluded resources from the backup identified by a given label
Additionally, you can schedule regular backups and manage the storage of backups and volume snapshots you create by specifying a retention period for each backup and deleting backups that are no longer needed.
vmware-system-tmcare not included in backups.
For more information about Velero, visit https://velero.io/docs.
For information on how to use the data protection features in Tanzu Mission Control, see Protecting Data in Using VMware Tanzu Mission Control.
About Backup Storage
For the storage of your backups, you can specify a target location that allows Tanzu Mission Control to manage the storage of backups, provisioning resources as necessary according to your specifications. However, if you prefer to manage your own storage for backups, you can also specify a target location that points to a storage location that you create and maintain in your cloud provider account, such as an AWS S3 or S3-compatible storage location or an Azure Blob storage location. With self-provisioned storage, you can leverage existing storage investments for backups, reducing network and cloud storage costs, and apply existing storage policies, quotas, and encryption. For a list of supported S3-compatible providers, see S3-Compatible object store providers in the Velero documentation.
- The data protection credential specifies the access credentials for the account where your backup is stored. This account can be either your AWS account where Tanzu Mission Control manages backup storage, or an account where you manage backups (the account that contains your AWS S3 or S3-compatible storage or the subscription that contains your Azure Blob storage).
- The data protection target location identifies the place where you want the backup stored, and references the associated data protection credential. You can share the target location across multiple cluster groups and clusters.
About Volume Backup
Tanzu Mission Control leverages Velero and supports backing up and restoring Kubernetes volumes. Velero Supports backing up and restoring volumes using the File System Backup (FSB) method and the Container Storage Interface (CSI) snapshot method.
File System Backup
Kubernetes volumes attached to pods can be backed up from the file system of the volumes. This approach is called File System Backup (FSB) or Pod Volume Backup. Tanzu Mission Control uses Velero with restic to achieve this.
FSB can be enabled while enabling data protection on the cluster. You can also enable or disable FSB from the data protection page on the cluster. On enabling FSB, restic gets installed on the cluster and Velero backs up all pod volumes using restic. Data on the volumes backed up using FSB, will be copied to the backup storage location using restic.
FSB offers two approaches of discovering pod volumes to be backup::
Opt-in approach: Every pod containing a volume to be backed up using FSB must be annotated with the volume’s name using the
Opt-out approach: All pod volumes are backed up using FSB, with the ability to opt-out any volumes that should not be backed up using the
backup.velero.io/backup-volumes-excludesannotation on the pod.
For more information about Velero, see Velero Backup.
If FSB is enabled on your cluster, FSB with Opt-out is the default setting for all backup operations. If FSB is enabled, during backup operation all volumes are evaluated based on the specified Opt-out or Opt-in approach to exclude or include volumes to be backed up using FSB.
Benefits of using FSB include:
It is capable of backing up and restoring almost any type of Kubernetes volume. Therefore, if you need a volume type that doesn’t have the concept of a native snapshot or CSI volume snapshot, FSB might be the best choice.
It is not tied to a specific storage platform, so you could save the backup data to a different storage platform from the one backing Kubernetes volumes, for example, a durable storage media.
FSB backs up data from the live file system, so the backup data is less consistent than the CSI volume snapshot approach.
It accesses the file system from the mounted hostpath directory, so the pods need to run as root user and even under privileged mode in some environments.
If both FSB and CSI volume snapshot are enabled, Velero will first backup volumes using FSB based on the specified approach (opt-in or opt-out). Volumes not included in FSB backup will be backed up by CSI volume snapshot if they meet prerequisites given in Requirements for CSI Volume Backup in Using Tanzu Mission Control. It is recommended to enable both FSB and CSI volume snapshots in case the cluster has volume types which do not support CSI volume snapshots. Such volumes can be backed up using FSB.
Container Storage Interface (CSI) Volume Snapshot
Velero supports backup and restore of CSI driver backed volumes using the Requirements for CSI Volume Backup in Using Tanzu Mission Control. To create a CSI snapshot, it requires a volume snapshot class that tells the Kubernetes engine which driver file to use when creating snapshots.
CSI snapshot is available only for persistent volumes created using CSI drivers supporting volume snapshot. Data on the CSI snapshots is not copied to the backup storage location. You should check with your cloud provider about the snapshot durability.
CSI Snapshot is not available for TKG clusters.
For more information about CSI prerequisites, see Requirements for CSI Volume Backup in Using Tanzu Mission Control.
FSB and CSI Usage
TMC allows you to enable and disable FSB and CSI snapshot independently.
If only FSB is enabled, Velero evaluates volumes to be backed up using FSB based on the specified approach (Opt-in or Opt-out). During restore, FSB backed up volumes will always be restored.
If both FSB and CSI volume snapshot are enabled and backup configured with CSI snapshot, Velero first evaluates volumes to be backed up using FSB based on the specified approach (Opt-in or Opt-out). CSI-driver based volumes not included in FSB backup are backed up by CSI volume snapshot if they meet the prerequisites stated in Requirements for CSI Volume Backup in Using Tanzu Mission Control.
If both FSB and CSI volume snapshot are enabled but backup is not configured with CSI snapshot, Velero will evaluate volumes to be backed up using FSB based on the specified approach (Opt-in or Opt-out). Volumes not included in FSB will not be backed up.
If only CSI Snapshot is enabled and backup configured with CSI snapshot, Velero will backup only CSI-driver based volumes if they meet prerequisites stated in Requirements for CSI Volume Backup. None of the non-CSI driver based volumes will be backed up.
If neither FSB nor CSI snapshot is enabled, no volumes will be backed up.
It is recommended to enable both FSB and CSI volume snapshot, otherwise if the cluster has non-CSI driver based volume types those volumes will not be backed up. In addition, you may use the Opt-in or Opt-out approach to exclude the CSI-based volumes from FSB so they can be backed up using CSI snapshot.
About Backup Restoration Between Different Clusters
When you create a backup using Tanzu Mission Control, that backup can be available for restoration to other clusters in your organization. This feature allows you to create a backup in one cluster and restore it to a different cluster, even clusters running on different platforms.
- A Kubernetes version downgrade (restoring to a cluster running a lower version of Kubernetes) can cause incompatibility of core API groups and other issues associated with feature availability. Use this approach judiciously.
- If a Kubernetes version upgrade (restoring to a cluster running a higher version of Kubernetes) causes incompatibility of core API groups, you must update the impacted custom resources in the source cluster prior to creating the backup.
networking.k8s.io/v1beta1API is no longer supported as of Kubernetes version 1.22.
You cannot restore a backup that contains restic volumes on cluster without restic. Additionally, you can restore a backup that contains volume snapshots to another cluster only if both clusters share the same cloud provider account.
- By default, persistent volume claims (PVCs) might fail to bind to volumes because the appropriate storage class from the source cluster doesn't exist in the target cluster. To make sure your volumes bind, use a storage class map as described in https://velero.io/docs/v1.8/restore-reference/#changing-pvpvc-storage-classes in the Velero documentation.
For example, the following configmap maps the
managed-premiumstorage classes from an AKS cluster to the
gp2storage class in an EKS cluster.
apiVersion: v1 kind: ConfigMap metadata: name: change-storage-class-config namespace: velero labels: velero.io/plugin-config: "" velero.io/change-storage-class: RestoreItemAction data: # Map the "default" and "managed-premium" storage classes backed by AzureDisk on # the source cluster to "gp2", a storage class backed by AWS EBS on the current # (destination) cluster. default: gp2 managed-premium: gp2
- Custom resources from the source cluster might not exist in the target cluster.
tiers.security.antrea.tanzu.vmware.comfrom a Tanzu Kubernetes cluster are not found in an AKS cluster.
You can exclude resources during restore to help avoid this issue.
- Resource differences between the source and target cluster might impact functionality.
Some packages install webhooks that can cause issues when the source and target are not the same cluster. For example,
validatingwebhookconfiguration.admissionregistration.k8s.iofrom an AKS will impact the functionality of an EKS cluster.
You can exclude resources during restore to help avoid this issue.