The Avi Kubernetes Operator (AKO) is an operator which works as an ingress Controller and performs functions specific to NSX Advanced Load Balancer in the TKGs environment with the NSX Advanced Load Balancer Controller. It remains in sync with the necessary TKGs objects and calls Controller APIs to configure the virtual services.

The AKO deployment consists of the following components:

  • The NSX Advanced Load Balancer Controller

  • The Service Engines (SE)

  • The Avi Kubernetes Operator (AKO)

An overview of the AKO deployment is shown below:



Note:

Security restrictions prevent AKO from communicating with an NSX Advanced Load Balancer Controller installed in the management domain where VCenter or NSX-Manager is located. The NSX Advanced Load Balancer Controller can be deployed anywhere, assuming AKO can route to it.

Deployment Guide

AKO can be installed on each workload cluster, using HELM to sync only ingress. Load Balancer sync must be deactivated on AKO because NSX-T LB cannot be turned off, and AKO and NCP will overwrite the LB VIP if both are managing the LB service.

Create Two Tier-1s for NSX Advanced Load Balancer

In NSX-T Manager, create a new dedicated Tier-1 Gateway for use with NSX Advanced Load Balancer Management. Also, create an additional Tier-1 Gateway for use as the NSX Advanced Load Balancer VIP Network. This will allow north-bound connections to T0 gateways and south-bound to the individual segments.



The architecture of the T1-Avi-Mgmt is shown above.



Ensure that the proper route advertisements are configured so that NSX Advanced Load Balancer can redistribute the VIP routes to the T0 gateway.

For detailed instructions on how to create this T1, see Installing NSX Advanced Load Balancer in VMware NSX-T Environments topic in the VMware NSX Advanced Load Balancer Installation Guide.

Create a Segment under each NSX Advanced Load Balancer Tier-1

The NSX Advanced Load Balancer requires a segment to be created under each of the Tier 1, one segment for Management and the other for Vip/Data.



For additional details, see Installing NSX Advanced Load Balancer in VMware NSX-T Environments topic in the VMware NSX Advanced Load Balancer Installation Guide.

Configure NSX-T Cloud on NSX Advanced Load Balancer

The point of integration in NSX Advanced Load Balancer, with any infrastructure, is called a cloud. For the NSX-T environment, an NSX-T cloud has to be configured. For more information on configuring an NSX-T cloud, see Installing NSX Advanced Load Balancer in VMware NSX-T Environments topic in the VMware NSX Advanced Load Balancer Installation Guide.

Set Default Gateway in NSX Advanced Load Balancer

Set Default Gateway for the T1 vrf in NSX Advanced Load Balancer. The DG needs to be set to allow outbound traffic from the NSX Advanced Load Balancer VIP/Data segment and the Management segment. See the following screenshot for an example. First, the Data/Vip DG is set in VMware NSX Advanced Load Balancer, as shown below:



Next, the Management DG needs to be set, if DHCP is not available. This is done using CLI.

Example commands:

shell 

switchto cloud nsxt 

terminal mode linux_command_line 

configure vrfcontext management --static_routes.1.prefix 0.0.0.0/0 --static_routes.1.next_hop 172.16.30.1 --static_routes.1.route_id 1

Configure VMware NSX Advanced Load Balancer IPAM Profile

The VMware NSX Advanced Load Balancer allocates IP addresses from a pool of IP addresses within the subnet configured as shown below. After creating this profile, Modify the NSX-T cloud and add this profile.



For more information, see Configuring NSX Advanced Load Balancer IPAM topic in the VMware NSX Advanced Load Balancer Installation Guide.

Configure VMware NSX Advanced Load Balancer DNS Profile

AKO needs to be authoritative in the domain that is specified below. After creating the profile, Modify the NSX-T cloud and add the profile.



For more details, see NSX Advanced Load Balancer IPAM and DNS topic in the VMware NSX Advanced Load Balancer Installation Guide.

Configure NSX-T DFW Rules

The default rule in WCP will deny all ingress traffic. Configure a DFW rule to allow SEs to the backend nodes/pods. Configure additional DFW rules to allow clients to VIPs and E-W SE-SE Traffic.





Deploy AKO through Helm

Configure the appropriate fields in the values.yaml file to allow AKO to integrate with VMware NSX Advanced Load Balancer + NSX-T.

  • layer7Only – Set to True

  • L7Settings.serviceType – Under L7Settings, set to NodePort

  • nsxtT1LR – Full path to the VMware NSX Advanced Load Balancer Tier 1. Example: “/infra/tier-1s/Avi-T1”

  • vipNetworkList – Set to the VMware NSX Advanced Load Balancer VIP Network

  • ControllerSettings – Set these for VMware NSX Advanced Load Balancer and the NSXT Cloud

  • avicredentials – Optional, can be passed through CLI

The full path for VMware NSX Advanced Load Balancer Tier1 in NSX-T can be found through an VMware NSX Advanced Load Balancer CLI command:



Installing AKO

helm install  ako/ako  --generate-name --version 1.7.1 -f /ako/values.yaml  --set avicredentials.username=admin --set avicredentials.password='password123' --namespace=avi-system

The above command will create a pod named: ako-0 in the avi-system namespace. Nothing will be created in VMware NSX Advanced Load Balancer until an ingress is created.

For detailed instructions on how to deploy AKO through helm, see Install Avi Kubernetes Operator.