AKO provides support for sharing VIP among multiple Kubernetes Services of type LoadBalancer deployed in the same namespace. Generally, with LoadBalancer services, AKO creates dedicated L4 virtual services in the NSX Advanced Load Balancer Controller, but multiple LoadBalancer services can also be combined to share a single VIP.

Shared VIP is supported for ClusterIP, NodePort, and NodePortLocal service types in the AKO configuration.

Sharing Single VIP

Sharing of single VIP among multiple services is achieved by providing an annotation to multiple LoadBalancer Services, where VIP sharing is intended. The annotation to be applied is ako.vmware.com/enable-shared-vip with a string value as shown below:

apiVersion: v1
kind: Service
metadata:
  annotations:
    ako.vmware.com/enable-shared-vip: "shared-vip-key-1"
  name: sharedvip-avisvc-lb1
  namespace: default
spec:
  type: LoadBalancer
  ports:
  - port: 80
    targetPort: 8080
  selector:
    app: avi-server
---
apiVersion: v1
kind: Service
metadata:
  annotations:
    ako.vmware.com/enable-shared-vip: "shared-vip-key-1"
  name: sharedvip-avisvc-lb2
  namespace: default
spec:
  type: LoadBalancer
  ports:
  - port: 80
    protocol: UDP
    targetPort: 8080
  selector:
    app: avi-server

AKO creates a single L4 virtual service (with a single VIP) based on the annotation value, and the port, protocol, app selector information will be used to configure pools and backend servers for this virtual service. After the successful creation of the corresponding Virtual Service and VIP, the Status of both the LoadBalancer Services will reflect the single VIP configured on the NSX Advanced Load Balancer Controller.

NAME                      TYPE           CLUSTER-IP       EXTERNAL-IP     PORT(S)        AGE
 sharedvip-avisvc-lb1      LoadBalancer   10.108.153.227   100.64.196.61   80:31658/TCP   6d23h
 sharedvip-avisvc-lb2      LoadBalancer   10.102.147.29    100.64.196.61   80:31331/UDP   6d23h

If there is a requirement to set a preferred static VIP through the .spec.loadBalancerIP field in the service, all LoadBalancer services sharing the annotation value must have the same preferred VIP provided in the spec. If two services under the same Annotation value have different static VIP set, no virtual service will be configured. This is treated as a misconfiguration and will be logged in AKO accordingly.

An example of configuring multiple LoadBalancer services to share a preferred VIP is shown below:

apiVersion: v1
 kind: Service
 metadata:
   annotations:
     ako.vmware.com/enable-shared-vip: "shared-vip-key-1"
   name: sharedvip-avisvc-lb1
   namespace: default
 spec:
   type: LoadBalancer
   loadBalancerIP: 100.64.196.75
   ports:
   - port: 80
     targetPort: 8080
   selector:
     app: avi-server
 ---
 apiVersion: v1
 kind: Service
 metadata:
   annotations:
     ako.vmware.com/enable-shared-vip: "shared-vip-key-1"
   name: sharedvip-avisvc-lb2
   namespace: default
 spec:
   type: LoadBalancer
   loadBalancerIP: 100.64.196.75
   ports:
   - port: 80
     protocol: UDP
     targetPort: 8080
   selector:
     app: avi-server

The expected status message must have the VIP matching the preferred static IP provided in the Service spec.

NAME                      TYPE           CLUSTER-IP       EXTERNAL-IP     PORT(S)        AGE
  sharedvip-avisvc-lb1      LoadBalancer   10.108.153.227   100.64.196.75   80:31658/TCP   6d23h
  sharedvip-avisvc-lb2      LoadBalancer   10.102.147.29    100.64.196.75   80:31331/UDP   6d23h
Note:

Consider the following when configuring the services with the aforementioned annotation:

  • Ensure that LoadBalancer services which are intended to share a VIP, must have the same annotation value. As shown in the example above, the annotation value shared-vip-key-1 is same for both services.

  • To avoid any errors while configuring the virtual service on the NSX Advanced Load Balancer Controller, it is required that there is no conflicting Port-Protocol pairs in the LoadBalancer services that share the annotation value. From the example, both services are exposing a unique, non-conflicting Port-Protocol for the backend application, namely, 80/TCP and 80/UDP.

  • The annotation must be provided only on service of type LoadBalancers.

An L4Rule CRD can also be used to specify a preferred IP for the LoadBalancers. For more information, see Express Custom Load Balancer IP. However, L4Rule cannot be used for services with shared vip if SSL termination is required to be enabled for the services.

AviInfrasetting Support

AviInfraSetting resources can be attached to LoadBalancer Kubernetes services using the annotation aviinfrasetting.ako.vmware.com/name: <aviinfra-crd-name>.

For more information, see AviInfraSetting.

Note:

Make sure that LoadBalancer services which are intended to share a VIP must have the same avinfrasetting annotation value.

L4Rule CRD Support

An L4Rule CRD can be attached to the Services of type LoadBalancer that are intended to share the VIP using the annotation ako.vmware.com/l4rule: <name-of-the-l4-rule-crd>. For more information on the L4Rule CRD, see L4 CRD Rule. However, there is an exception if SSL termination is required to be enabled for the services. So, if enableSsl is set to true for any port in listenerProperties section of the L4Rule, then that L4Rule should only be applied to a single LoadBalancer service. This exception is because a virtual service of type L4 SSL can have only one back end pool configured.