Use this reference when configuring additional parameters of load-balancer-and-ingress-service addon or managing AKO objects(aviinfrasetting, gatewayclass, gateway) via the Custom Resources(CRs) tab.

Configurable parameters

Note:

Some parameters are only applicable for certain topology(e.g. NSX-T environment) or certain feature(e.g. Provide cluster control plane HA with Avi). Customize these parameters carefully base on your actual environment.

Parameter

description

type

Default value

Note

cloudName

Cloud name configured in Avi Controller

string

Mandatory, formatted on UI

controllerVersion

Avi Controller version

string

20.1.3

controlPlaneNetwork.cidr

ControlPlaneNetwork.cidr describes the control plane network cidr of the cluster

string

Only for using Avi provide control plane HA feature

controlPlaneNetwork.name

ControlPlaneNetwork.name describes the control plane network name of the cluster

string

Only for using Avi provide control plane HA feature

defaultServiceEngineGroup

Service engine group name configured in Avi Controller

string

Mandatory, formatted on UI

defaultVipNetwork

VIP network name in Avi Controller

string

Mandatory, formatted on UI

defaultVipNetworkCidr

VIP network CIDR in Avi Controller

string

Mandatory, formatted on UI

defaultVipNetworkIpPools.end

End represents the ending IP address of the pool

string

defaultVipNetworkIpPools.start

Start represents the starting IP address of the pool

string

defaultVipNetworkIpPools.type

Type represents the type of IP Address

enum["V4"]

V4

extraConfigs.apiServerPort

ApiServerPort specifies Internal port for AKO's API server for the liveness probe of the AKO pod

integer

8080

extraConfigs.disableStaticRouteSync

DisableStaticRouteSync describes AKO should sync static routing or not. If the POD networks are reachable from the Avi SE, this should be to true. Otherwise, it should be false.

boolean

false

extraConfigs.enableEvents

Defines enable or disable event broadcasting via AKO

boolean

false

extraConfigs.enableEVH

EnableEVH specifies if you want to enable the Enhanced Virtual Hosting Model in Avi Controller for the Virtual Services

boolean

false

extraConfigs.fullSyncFrequency

FullSyncFrequency controls how often AKO polls the Avi controller to update itself with cloud configurations.

string

1800

extraConfigs.ingress.defaultIngressController

Enabling this flag will use AKO as the default ingress controller

boolean

false

extraConfigs.ingress.disableIngressClass

DisableIngressClass will prevent AKO Operator to install AKO IngressClass into workload clusters

boolean

true

extraConfigs.ingress.enableMCI

Enabling this flag would tell AKO to start processing multi-cluster ingress objects

boolean

false

extraConfigs.ingress.nodeNetworkList.cidrs

Cluster node network cidrs

string list

Mandatory when extraConfigs.ingress.serviceType is ClusterIP, formatted on UI

extraConfigs.ingress.nodeNetworkList.name

Cluster node network name

string

Mandatory when extraConfigs.ingress.serviceType is ClusterIP, formatted on UI

extraConfigs.ingress.noPGForSNI

NoPGForSNI describes if you want to get rid of poolgroups from SNI VSes. Do not use this flag if you don't want http caching

boolean

false

extraConfigs.ingress.passthroughShardSize

PassthroughShardSize controls the passthrough virtualservice numbers

enum["SMALL", "MEDIUM", "LARGE"]

SMALL

extraConfigs.ingress.serviceType

ServiceType describes ingress methods for a service

enum["ClusterIP", "NodePort", "NodePortLocal"]

ClusterIP

Mandatory, formatted on UI

extraConfigs.ingress.shardVSSize

ShardVSSize describes ingress shared virtual service size

enum["SMALL", "MEDIUM", "LARGE", "DEDICATED"]

SMALL

extraConfigs.l4Config.autoFQDN

AutoFQDN controls the FQDN generation. Valid value should be default(<svc>.<ns>.<subdomain>), flat (<svc>-<ns>.<subdomain>) or disabled

enum["default", "flat", "disabled"]

disabled

extraConfigs.l4Config.defaultDomain

DefaultDomain controls the default sub-domain to use for L4 VSes when multiple sub-domains are configured in the cloud.

string

extraConfigs.layer7Only

Layer7Only specifies if you want AKO only to do layer 7 load balancing

boolean

false

extraConfigs.log.logFile

LogFile specifies the log file name

string

extraConfigs.log.logLevel

LogLevel specifies the AKO pod log level

enum["INFO", "DEBUG", "WARN", "ERROR"]

INFO

extraConfigs.log.mountPath

MountPath specifies the path to mount PVC

string

extraConfigs.log.persistentVolumeClaim

PersistentVolumeClaim specifies if a PVC should make for AKO logging

string

extraConfigs.namespaceSelector.labelKey

NameSpaceSelector.labelKey contains label key used for namespace migration. Same label key has to be present on namespace/s which needs migration/sync to AKO

string

extraConfigs.namespaceSelector.labelValue

NameSpaceSelector.labelValue contains label value used for namespace migration. Same label value has to be present on namespace/s which needs migration/sync to AKO

string

extraConfigs.networksConfig.bgpPeerLabels

BGPPeerLabels specifies BGP peers, this is used for selective VsVip advertisement.

string list

extraConfigs.networksConfig.enableRHI

EnableRHI specifies cluster wide setting for BGP peering.

boolean

false

extraConfigs.networksConfig.nsxtT1LR

T1 Logical Segment mapping for backend network.

string

Only applies to NSX-T cloud.

extraConfigs.nodePortSelector.key

NodePortSelector only applicable if serviceType is NodePort

string

extraConfigs.nodePortSelector.value

NodePortSelector only applicable if serviceType is NodePort

string

extraConfigs.primaryInstance

Defines if the AKO instance is primary. Value `true` indicates that AKO instance is primary. In a multiple AKO deployment in a cluster, only one AKO instance should be primary

boolean

true

extraConfigs.rbac.pspEnabled

PspEnabled enables the deployment of a PodSecurityPolicy that grants AKO the proper role

boolean

false

extraConfigs.rbac.pspPolicyAPIVersion

PspPolicyAPIVersion decides the API version of the PodSecurityPolicy

string

extraConfigs.servicesAPI

ServicesAPI specifies if enables AKO in services API mode: https://kubernetes-sigs.github.io/service-apis/. Currently, implemented only for L4. This flag uses the upstream GA APIs which are not backward compatible with the advancedL4 APIs which uses a fork and a version of v1alpha1pre1

boolean

true

extraConfigs.vipPerNamespace

Enabling this flag would tell AKO to create Parent VS per Namespace in EVH mode

boolean

false

tenant.context

Context is the type of AVI tenant context.

enum["Provider", "Tenant"]

Provider

This field is immutable

tenant.name

Name is the name of the tenant.

string

This field is immutable

workloadCredentialRef.name

WorkloadCredentialRef points to a Secret resource that includes the username and the password to access and configure the AviController.

  * username Username used with basic authentication for the Avi REST API  * password Password used with basic authentication for the Avi REST API

This field is optional. When it's not specified, username/password will beautomatically generated for each Cluster and Tenant needs to be non-nil in this case.

string

workloadCredentialRef.namespace

The namespace of the Secret resource includes the username and password

string

A simplest CR sample is:

metadata:
  name: load-balancer-and-ingress-service
  clusterName: wc0
spec:
  name: load-balancer-and-ingress-service
  clusterRef:
    name: wc0
    namespace: wc0
  config:
    stringData:
      values.yaml: |
        cloudName: vcenter-cloud0
        defaultServiceEngineGroup: wc0-se-group
        defaultVipNetwork: oam-vip-dvpg
        defaultVipNetworkCidr: 172.16.73.0/24
        extraConfigs:
          ingress:
            serviceType: ClusterIP
            nodeNetworkList:
              - networkName: cluster-mgmt-dvpg
                cidrs:
                  - 172.16.68.0/22

Managing AKO objects via load-balancer-and-ingress-service add-on

Append aviObjects section to load-balancer-and-ingress-service add-on CR to manage AKO objects(aviinfrasetting, gatewayclass, gateway) lifecycle.

A sample CR with aviObjects is:

metadata:
  name: load-balancer-and-ingress-service
  clusterName: wc0
spec:
  name: load-balancer-and-ingress-service
  clusterRef:
    name: wc0
    namespace: wc0
  config:
    stringData:
      values.yaml: |
        cloudName: vcenter-cloud0
        defaultServiceEngineGroup: wc0-se-group
        defaultVipNetwork: oam-vip-dvpg
        defaultVipNetworkCidr: 172.16.73.0/24
        extraConfigs:
          ingress:
            serviceType: ClusterIP
            nodeNetworkList:
              - networkName: cluster-mgmt-dvpg
                cidrs:
                  - 172.16.68.0/22
        aviObjects:
          aviinfrasettings:
            - metadata:
                name: ais0
              spec:
                seGroup:
                  name: wc0-se-group
                network:
                  vipNetworks:
                    - networkName: oam-vip-dvpg
                l7Settings:
                  shardSize: MEDIUM
            - metadata:
                name: ais1
              spec:
                seGroup:
                  name: wc0-se-group
                network:
                  vipNetworks:
                    - networkName: sig-vip-dvpg
                l7Settings:
                  shardSize: MEDIUM
          gatewayclasses:
            - metadata:
                name: gwc0
              spec:
                controller: ako.vmware.com/avi-lb
                parametersRef:
                  group: ako.vmware.com
                  kind: AviInfraSetting
                  name: ais0
          gateways:
            - metadata:
                name: gw0
                namespace: gw0
              spec:
                gatewayClassName: gwc0
                listeners:
                  - protocol: TCP
                    port: 80
                    routes:
                      selector:
                        matchLabels:
                          ako.vmware.com/gateway-namespace: gw0
                          ako.vmware.com/gateway-name: gw0
                      group: v1
                      kind: Service
                  - protocol: TCP
                    port: 8081
                    routes:
                      selector:
                        matchLabels:
                          ako.vmware.com/gateway-namespace: gw0
                          ako.vmware.com/gateway-name: gw0
                      group: v1
                      kind: Service

  • In this sample CR, two aviinfrasetting objects ais0ais1, one gatewayclass object gwc0, and one gateway object gw0 will be created or updated, if already exist.

  • Aviinfrasetting objects can be created with enableRhi: true and bgpPeerLabels as needed.

  • Edit load-balancer-and-ingress-service add-on and then switch to the Custom Resources(CRs) tab. Remove the specific AKO objects from aviObjects section to delete them from workload cluster.

  • TCA will create namespace(if not exist) for gateway objects but will not delete the namespace when deleting the gateway objects.