Multiple AKO instances can be deployed in a cluster to create namespace based isolation. This will allow AKO to operate over a group of Kubernetes namespaces, in order to handle objects from these namespaces only.

To run multiple AKO, the following features are used:

  1. Namespace Sync Feature

  2. AKO installation in User Provided Namespace

  3. Configuring Multiple AKO Instances

  4. Primary AKO installation

  5. Non-Primary AKO Installation

  6. Avi Object Name Convention

Namespace Sync Feature

The namespace sync feature allows Kubernetes/OpenShift objects from specific namespaces to be synced with AKO based NSX Advanced Load Balancer Controller. For that, namespace has to be labelled with same key:value pair as the labelKey and labelValue mentioned in values.yaml file. For more information on namespace sync feature, see Namespace Sync in AKO.

AKO installation in User Provided Namespace

While installing AKO, using helm, flag -n or --namespace can be used to specify namespace in which AKO has to be installed. If this flag is not specified, AKO will be installed in the avi-system namespace.

Configuring Multiple AKO Instances

In multiple AKO deployment, one AKO instance works as primary AKO instance. This AKO instance is responsible for VRF, static route configuration apart from syncing up Kubernetes/OpenShift objects from set of namespaces.

The flag primaryInstance present in values.yaml denotes whether AKO instance is primary or not. This flag takes boolean true/false value. True indicates AKO instance is primary.

Note:

  • In multiple AKO deployment, only one AKO instance must be primary.

  • Each AKO must be deployed in a different namespace.

Primary AKO installation

In the following example, primary AKO instance is running in avi-system namespace with namespace sync filter app: migrate. So this AKO will sync up Kubernetes/OpenShift objects from namespaces who has labels app: migrate.

helm install  ako/ako  --generate-name --version 1.10.1 -f /path/to/values.yaml  --set ControllerSettings.controllerHost=<controller IP or Hostname> --set avicredentials.username=<avi-ctrl-username> --set avicredentials.password=<avi-ctrl-password> --set AKOSettings.namespaceSelector.labelKey="app" --set AKOSettings.namespaceSelector.labelValue="migrate" --set AKOSettings.primaryInstance=true --namespace=avi-system

The helm install command without primaryInstance parameter will deploy primary AKO instance.

Non-Primary AKO Installation

In the following example, non-primary AKO instance is running in blue namespace with namespace sync filter key: value2. So this AKO will sync up Kubernetes/OpenShift objects from namespaces who has labels key: value2.

 helm install  ako/ako  --generate-name --version 1.10.1 -f /path/to/values.yaml  --set ControllerSettings.controllerHost=<controller IP or Hostname> --set avicredentials.username=<avi-ctrl-username> --set avicredentials.password=<avi-ctrl-password> --set AKOSettings.namespaceSelector.labelKey="key" --set AKOSettings.namespaceSelector.labelValue="value2" --set AKOSettings.primaryInstance=false --namespace=blue

Few things that to be considered in multiple AKO instances deployment:

  1. All AKO instances must interact with same NSX Advanced Load Balancer Controller

  2. Each Kubernetes/OpenShift namespace must be handled by one AKO only

  3. All AKO must be deployed either in ClusterIP or NodePort or NodePortLocal mode

Avi Object Name Convention

  1. For non-primary AKO instance, naming convention for shared VS is: Shared-VS-Name = <cluster-name>–<AKO-namespace>-Shared-L7-<Shard number>. Here <AKO-namespace> is namespace in which AKO pod is deployed.

  2. Non-primary AKO instance will create objects with username = ako-<cluster-name>-<AKO-namespace>