Migrating from Cloud Connector to AKO

Cloud connector (CC) is not supported from NSX Advanced Load Balancer Controller version 20.1 onwards. To migrate the existing workload present in a Kubernetes cluster from NSX Advanced Load Balancer Controller running the OpenShift/Kubernetes cloud using cloud connector to AKO based NSX Advanced Load Balancer Controller, following two features are used:

Namespace-Driven Inclusion/Exclusion of OpenShift/Kubernetes Applications

For migration, the exclusion feature allows ingresses/routes from specific namespace(s) to be deleted (excluded) from the NSX Advanced Load Balancer Controller. For that, the namespace has to be labelled with the same key:value pair as that of exclude attribute and exclude value mentioned in cloud. This feature is supported in NSX Advanced Load Balancer Controller running the OpenShift/Kubernetes cloud using cloud connector.

The Namespace Sync feature

The namespace sync feature allows Kubernetes objects from specific namespace to be synced with an AKO based NSX Advanced Load Balancer Controller. For that, the namespace has to be labelled with same key:value pair as that labelKey and labelValue mentioned in values.yaml file.

For more information, see Namespace Sync in AKO.

Environments

This migration activity is currently supported for vCenter cloud with Kubernetes based workloads.

Workflow

This section explains the steps of migration

Pre-requisites

• Setup a new Controller (compatible with AKO version 1.4.1 and above)

• Setup a new vCenter cloud with write access

• Setup an IPAM on vCenter cloud. It is recommended to configure IPAM on vCenter cloud with non-overlapping usable network to avoid IP address conflict or non-availability of IP address after migration

• Setup a DNS Service on vCenter cloud

• Replicate the NSX Advanced Load Balancer side objects, referred by Ingresses/Services as part of AVI_PROXY annotations, to the new Controller.

• Ensure the AKO version is 1.4.1 and above.

Deploying AKO CRDs

For each AVI_PROXY annotation present in an ingress, create the corresponding AKO Http rule/Host rule/ AviInfrasetting rule. For more information, see Setting up Routing Rules using CRDs.

If any AVI_PROXY annotation is not supported by these CRDs, then the virtual service which is migrated to AKO based NSX Advanced Load Balancer Controller will not have same features as that as the NSX Advanced Load Balancer Controller running the OpenShift/Kubernetes cloud using cloud connector.

Note:

List of AVI_PROXY annotations supported by AKO CRDs will be published soon.

Deploying AKO

With AKO, the SEs are deployed outside the Kubernetes cluster. So deploy AKO with static routing set to true or false depending upon the POD network reachability and with namespace sync feature enabled.

AKOSettings:
  .
  .
  # NamespaceSelector contains label key and value used for namespacemigration
  # Same label has to be present on namespace/s which needs migration/sync to AKO
  namespaceSelector:
    labelKey: "app"
    labelValue: "migrate"
  .
  .

As an example, AKO is deployed with app as a key and migrate as a value for namespace selector. So AKO will sync up all objects from namespace(s) which has this label and corresponding NSX Advanced Load Balancer objects will be created in the NSX Advanced Load Balancer Controller.

Excluding Namespace-Driven OpenShift/Kubernetes Applications

Set an exclude attribute and exclude value for Openshift/Kubernetes cloud either from the NSX Advanced Load Balancer UI or CLI.

The key and value should be same as mentioned in values.yaml of AKO.

• Set up an exclude attribute and exclude value using the UI as shown below:

  
  
  

• Set up an exclude attribute and exclude value using NSX Advanced Load Balancer shell:

  [admin:cc-controller-host-name]: > configure cloud Default-Cloud
  [admin:cc-controller-host-name]: cloud> oshiftk8s_configuration
  [admin:cc-controller-host-name]: cloud:oshiftk8s_configuration> ns_exclude_attributes
  [admin:cc-controller-host-name]: cloud:oshiftk8s_configuration:ns_exclude_attributes> attribute app
  [admin:cc-controller-host-name]: cloud:oshiftk8s_configuration:ns_exclude_attributes> value migrate
  [admin:cc-controller-host-name]: cloud:oshiftk8s_configuration:ns_exclude_attributes> save
  [admin:cc-controller-host-name]: cloud:oshiftk8s_configuration> save
  [admin:cc-controller-host-name]: cloud> save

In this example, the exclude attribute is set as app and the exclude value is set as migrate. This deletes virtual services of namespace(s), which has same label, from the NSX Advanced Load Balancer Controller running the OpenShift/Kubernetes cloud using cloud connector.

Setting Kubernetes Namespace Label

Label namespace(s) in a Kubernetes cluster with the same key:value pair.

apiVersion: v1
kind: Namespace
metadata:
  .
  labels:
    app: migrate
  name: red
  .
.

As shown in the example, the red namespace is labelled with app: migrate. This will result in deletion of virtual services of red namespace from the NSX Advanced Load Balancer Controller running the OpenShift/Kubernetes cloud using the cloud connector and creation of new virtual services for red namespace in AKO-based NSX Advanced Load Balancer Controller.

Note:

There will be traffic disruption during migration.

Internal Testing

The newly created objects in an AKO-based NSX Advanced Load Balancer Controller may get new VIPs. Hence, the internal traffic tests need to be performed for migrated applications.

After successful testing, migrate objects from other namespaces by labelling the namespace one by one.

Redirecting Client Traffic

After migrating all the L4-L7 applications to an AKO-based NSX Advanced Load Balancer Controller, the client traffic can be redirected to new VIPs by one of the following ways:

• Change DNS server entry in corporate DNS server to point to new DNS service on the AKO-based NSX Advanced Load Balancer Controller.

• Change IP address of DNS service running on AKO based NSX Advanced Load Balancer Controller with IP address of DNS service present on NSX Advanced Load Balancer Controller running the OpenShift/Kubernetes cloud using the cloud connector if IPAM from both clouds have overlapping subnets.

Cleaning Up of Cloud Connector (OpenShift Cloud)

After migrating all the Kubernetes objects, the old OpenShift/Kubernetes cloud can be deleted.