Starting with AKO version 1.12.1, Gateway API v1 release is supported. The field GatewayAPI under the section featureGates in values.yaml must be set to true to enable this feature. This generates a new container within the AKO pod that will handle the Gateway API related objects.

AKO supports the following objects in the Gateway API:
  1. GatewayClass (v1)
  2. Gateway (v1)
  3. HTTPRoute (v1)
Note:

AKO currently supports all the fields that are mentioned as Support: Core in the above objects. For more details, see limitations.

Table 1. Support Matrix
Release GatewayClass Gateway HTTPRoute GRPCRoute TLSRoute TCPRoute UDPRoute

ReferenceGrant
1.12.1 v1 v1 v1 Not Supported Not Supported Not Supported Not Supported

Not Supported

Installation

Follow the below steps to enable the GatewayAPI feature in AKO:

  1. Set the GatewayAPI under the section featureGates to true in the values.yaml.

  2. Do a helm install or upgrade using the edited values.yaml to install or upgrade the AKO.

A GatewayClass avi-lb with controllerName as ako.vmware.com/avi-lb will get installed as part of the installation. An Infrastructure Provider can request the cluster operators to use this GatewayClass in their Gateway objects so that the AKO honours the objects created by them.
Note:

The GatewayClass, Gateway, and Route CRD definitions must be installed on the cluster before enabling the GatewayAPI feature in AKO. For more information on CRDs, see gateway-api.

Gateway API Objects

GatewayClass

GatewayClass aggregates a group of Gateway objects, similar to how IngressClass aggregates a group of Ingress objects. GatewayClasses formalizes types of load balancing implementations, which can be different for different load-balancing vendors (Avi/Nginx/HAProxy, and so on).

AKO identifies GatewayClasses that point to ako.vmware.com/avi-lb as the .spec.controllerName value in the GatewayClass object.

A sample GatewayClass object looks as shown below:

apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: avi-lb
spec:
  controllerName: "ako.vmware.com/avi-lb"

It is important that the .spec.controllerName value specified must match ako.vmware.com/avi-lb for AKO to honour the GatewayClass and corresponding Gateway API objects.

Note:

A GatewayClass named avi-lb will get installed as part of the helm install or upgrade when the user enables the Gateway API feature.

Gateway

The Gateway object represents an instance of a service-traffic handling infrastructure by binding Listeners to a set of IP addresses. The AKO validates the Gateway object and updates the status as Accepted. Then, the AKO translates the Gateway and its configuration as a Parent virtual service. The listeners in Gateway will be translated as listeners in the Parent virtual service, and secrets will be configured as Certificates in the Avi Load Balancer Controller and will be referenced in the same Parent virtual service. AKO updates the status of Gateway as programmed with the VIP of the Parent virtual service once the virtual service creation is completed.

The AKO created parent virtual service uses the name convention: ako-gw-<cluster-name>--<namespace of the gateway>-<name of the gateway>-EVH.

A sample Gateway object is shown below:

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: my-gateway
spec:
  gatewayClassName: avi-lb
  listeners:
  - name: foo-http
    protocol: HTTP
    port: 80
    hostname: *.example.com
  - name: bar-https
    protocol: HTTPS
    port: 443
    hostname: *.example.com
    tls:
      certificateRefs:
      - kind: Secret
        group: ""
        name: bar-example-com-cert

The above Gateway object would correspond to a single Layer 7 virtual service in the Avi Load Balancer Controller, with two ports (80, 443) exposed and a sslKeyAndCertificate created based on the secret bar-example-com-cert.

The hostname field .spec.listeners[i].hostname is mandatory. It can be configured with or without a wildcard, but cannot be only *.

Note:

Starting with version 1.11.1, AKO only supports HTTP and HTTPS as protocol and Secret kind for certificateRefs.

Users can also configure a user-preferred static IPv4 address in the Gateway Object using the .spec.addresses field, as shown below. This would configure the Layer 7 virtual service with a static IP, as mentioned in the Gateway Object.

spec:
  addresses:
  - type: IPAddress
    value: 10.1.1.10
Note:

Starting with AKO version 1.11.1, a single address of type IPAddress is supported. The Gateway must be re-created to update the address.

HTTPRoute

The HTTPRoute object provides a way to route HTTP requests. The AKO models a child virtual service based on this object. Starting with version 1.11.1, AKO supports match requests based on the hostname, path, and header specified. The filters to specify additional processing of the requests will be added as policy in the child virtual service by the AKO. The filters of type RequestHeaderModifier, RequestRedirect, and ResponseHeaderModifier are supported.

A sample HTTPRoute object is as shown below:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: my-http-app
spec:
  parentRefs:
  - name: my-gateway
  hostnames:
  - "foo.example.com"
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /bar
    backendRefs:
    - name: my-service1
      port: 8080
  - filters:
    - type: RequestHeaderModifier
      requestHeaderModifier:
        add:
          - name: my-header
            value: foo
    matches:
    - headers:
      - type: Exact
        name: magic
        value: foo
      path:
        type: PathPrefix
        value: /foo
    backendRefs:
    - name: my-service2
      port: 8080
      weight: 1
    - name: my-service3
      port: 8081
      weight: 2

The above HTTPRoute object gets translated to two child virtual service in the Avi Load Balancer Controller. One child virtual service with match criteria as the path begins with /bar and a single Pool Group with a single pool and another child virtual service with match criteria as path begins with /foo, a single Pool Group with two pools, and an HTTP Request policy to add my-header to the HTTP request forwarded to the back-ends.

Hostnames are mandatory and cannot contain wildcard.

Note:

AKO Gateway APIs does not support filters within backendRefs.

Gateway must be created before an HTTPRoute is created. If Gateways are created after creating HTTPRoute, then the HTTPRoute needs to be updated to trigger the informer.

Gateway API Objects to Avi Load Balancer Controller Objects Mapping

In AKO Gateway API Implementation, Gateway objects correspond to the following Controller objects:

  1. Gateway translates to an EVH Parent Virtual Service with port/protocol from each Listener added as the Service within parent virtual service.

  2. The tls specification (certificate refs) from each Gateway Listener will get added to the parent virtual service as SSLKeyAndCertificateRef.

  3. Every Secret created corresponds to an SSLKeyAndCertificate object.

  4. Addresses in a Gateway specification get added as static IPs for Vsvip for parent virtual service.

  5. Every Rule in HTTPRoute corresponds to an EVH Child Virtual Service, with Match translated to VH match and Filters translated to HTTPPolicySet configuration.

  6. Each backendRefs specification (list of backends) in an HTTPRoute Rule is added as a Pool Group.

  7. Each backendRef in an HTTPRoute Rule is translated to a pool.

Naming Conventions

AKO Gateway Implementation follows the following naming convention:

  1. ParentVS ako-gw-<cluster-name>--<gateway-namespace>-<gateway-name>-EVH

  2. ChildVS ako-gw-<cluster-name>–-<sha1 hash of <gateway-namespace>-<gateway-name>-<route-namespace>-<route-name>-<stringified FNV1a_32 hash of bytes(jsonified match)>>

  3. Pool ako-gw-<cluster-name>--<sha1 hash of <gateway-namespace>-<gateway-name>-<route-namespace>-<route-name>-<stringified FNV1a_32 hash of bytes(jsonified match)>-<backendRefs_namespace>-<backendRefs_name>-<backendRefs_port>>

  4. PoolGroup ako-gw-<cluster-name>–-<sha1 hash of <gateway-namespace>-<gateway-name>-<route-namespace>-<route-name>-<stringified FNV1a_32 hash of bytes(jsonified match)>>

  5. SSLKeyAndCertificate ako-gw-<cluster-name>--<sha1 hash of ako-gw-<cluster-name>--<hostname>

HTTP Traffic Splitting

AKO Gateway supports the Canary and Blue-Green traffic rollout.

For more information on configurations corresponding to this, see HTTP traffic splitting.

Status of Gateway API objects

AKO updates the status of all Gateway API objects with proper reasons. A typical status consists of a reason for the acceptance or rejection using which a user can debug the Gateway API object configuration.

A sample status of the Gateway is as shown below:

status:
  addresses:
  - type: IPAddress
    value: 10.1.1.12
  conditions:
  - lastTransitionTime: "2023-08-31T15:04:14Z"
    message: Gateway configuration is valid
    observedGeneration: 1
    reason: Accepted
    status: "True"
    type: Accepted
  - lastTransitionTime: "2023-08-31T15:04:16Z"
    message: Virtual service configured/updated
    observedGeneration: 1
    reason: Programmed
    status: "True"
    type: Programmed
  listeners:
  - attachedRoutes: 1
    conditions:
    - lastTransitionTime: "2023-08-31T15:07:35Z"
      message: Listener is valid
      observedGeneration: 1
      reason: Accepted
      status: "True"
      type: Accepted
    name: http
    supportedKinds:
    - group: gateway.networking.k8s.io
      kind: HTTPRoute

A sample HTTPRoute status is as shown below:

status:
  parents:
  - conditions:
    - lastTransitionTime: "2023-09-06T11:49:06Z"
      message: Parent reference is valid
      observedGeneration: 1
      reason: Accepted
      status: "True"
      type: Accepted
    controllerName: ako.vmware.com/avi-lb
    parentRef:
      group: gateway.networking.k8s.io
      kind: Gateway
      name: my-gateway
      namespace: default
      sectionName: http

Conditions and Caveats

  1. Gateway Limitations

    AKO accepts the following Gateway configuration:

    1. Gateway must contain at least one listener configuration in it.

    2. Gateway must not contain protocols other than HTTP or HTTPS.

    3. Gateway must contain a hostname. Hostname as * is not supported, and *.domain is supported.

    4. Gateway must not contain TLS modes other than Terminate.

    5. If the Secret specified in Gateway > listeners > TLSConfig does not exist or is in a namespace other than the Gateway namespace, AKO Gateway implementation will attach SystemDefaultCert to the Gateway. Once the Secret is created in the same namespace as that of Gateway, the certRef is updated accordingly.

    6. Gateway can have multiple listeners with the same or overlapping hostname.

    7. Two Gateways must not have listeners with the same or overlapping hostname.

    8. Gateway > listeners > allowedRoutes > namespaces > from with value selector, and thus Gateway > listeners > allowedRoutes > namespaces > selector is not supported.

  2. HTTPRoute Limitations

    AKO accepts the following HTTPRoute configuration:

    1. HTTPRoute must contain at least one parent reference.

    2. HTTPRoute must not contain * as hostname.

    3. HTTPRoute must not contain * in hostname.

    4. HTTPRoute must contain at least one hostname match with parent Gateway.

    5. Filters nested inside BackendRefs are not supported, that is, HTTPRoute > spec > rules > backendRefs > filters are not supported, whereas HTTPRoute > spec > rules > filters is supported.

  3. Resource Creation

    AKO imposes a restriction on the order of GatewayAPI object creation. An object that is referenced must be created first. For example, GatewayClass must be created before Gateway and Gateway before HTTPRoute creation. This restriction is only applicable to the Gateway API objects.

  4. High Availability Support

    AKO in active stand-by mode does not support Gateway APIs. Avi Load Balancer continues to deliver highly available L4 and L7 LBs for Gateway APIs.

  5. Configuring Static IP address

    The AKO supports Gateway objects with a single IPv4 address. The user can configure their preferred static IPv4 address by specifying spec.addresses in the Gateway object.

    A sample configuration is as shown below:

    spec:
  addresses:
  - type: IPAddress
    value: 10.1.1.10
    Note:

    The address of type IPAddress is only supported. The length of the addresses is also limited to a single address.

  6. Kubernetes Service types

    AKO Gateway API supports ClusterIP and NodePort Service types.

  7. Container Network Interface (CNI) providers

    AKO Gateway API supports Calico as a CNI provider.

  8. Platforms/ Environments supported

    AKO Gateway API is supported on the Kubernetes platform.

  9. Cloud Service Providers

    AKO Gateway API supports VMware vCenter/vSphere ESX Cloud.