Commercial route filters for traffic control for SCG for K8s

The open-source Spring Cloud Gateway project includes a number of built-in filters for use in Gateway routes. The following commercial filters provided by VMware Spring Cloud Gateway for Kubernetes can be used in addition to those included in the OSS project.

`AllowedRequestCookieCount`

This filter controls the maximum number of cookies allowed on a request. The filter will respond with a 431 Request Header Fields Too Large error if the number of cookies on a request exceeds the configured integer maximum.

In this example, only requests with two or fewer cookies will be accepted:

apiVersion: "tanzu.vmware.com/v1"
kind: SpringCloudGatewayRouteConfig
metadata:
  name: my-gateway-routes
spec:
  service:
    name: myapp
  routes:
    - ssoEnabled: true
      predicates:
        - Path=/api/**
      filters:
        - AllowedRequestCookieCount=2

`AllowedRequestHeadersCount`

This filter controls the maximum number of headers allowed on a request. The filter will respond with a 431 Request Header Fields Too Large error if the number of headers on a request exceeds the configured integer maximum.

In this example, only requests with four or fewer headers will be accepted:

apiVersion: "tanzu.vmware.com/v1"
kind: SpringCloudGatewayRouteConfig
metadata:
  name: my-gateway-routes
spec:
  service:
    name: myapp
  routes:
  - ssoEnabled: true
    predicates:
      - Path=/api/**
    filters:
      - AllowedRequestHeadersCount=4

`AllowedRequestQueryParamsCount`

This filter controls the maximum number of query parameters allowed on a request. The filter will respond with a 414 URL Too Large error if the number of query parameters on a request exceeds the configured integer maximum.

In this example, only requests with three or fewer query parameters will be accepted:

apiVersion: "tanzu.vmware.com/v1"
kind: SpringCloudGatewayRouteConfig
metadata:
  name: my-gateway-routes
spec:
  service:
    name: myapp
  routes:
    - ssoEnabled: true
      predicates:
        - Path=/api/**
      filters:
        - AllowedRequestQueryParamsCount=3

`CircuitBreaker`

The CircuitBreaker filter will reroute requests to a fallback destination when the intended destination responds with an HTTP error code.

The circuit breaker pattern is a useful approach to bolster resilience and prevent cascading failures in distributed systems. It works by preventing struggling services from becoming overloaded with failing requests, and thereby giving them chance to recover.

In electrical systems, a circuit breaker is a switch that triggers on a detected fault, isolating the faulty circuit and preventing further damage. This act is known as opening the circuit breaker (breaking the circuit). When the fault is corrected, the circuit breaker is closed (completing the circuit), allowing current to flow again. The circuit breaker pattern borrows this highly useful technique and terminology and applies it to distributed software systems.

Because our systems must be able to recover autonomously, there is also a half-open state which allows the circuit breaker to detect recovery of the downstream system. Following a timeout after the original detected failure, the circuit breaker will allow trial requests through to the destination system and then either fully close, or fully open and restart the timeout, based on the status of the response.

In the following example, the CircuitBreaker filter is used to cause requests to /api/** to fall back to /inCaseOfFailureUseThis, should the myapp service respond with an error code:

apiVersion: "tanzu.vmware.com/v1"
kind: SpringCloudGatewayRouteConfig
metadata:
  name: myapp-route-config
spec:
  service:
    name: myapp
  routes:
  - predicates:
      - Path=/api/**
    filters:
      - CircuitBreaker=myCircuitBreaker,forward:/inCaseOfFailureUseThis

The CircuitBreaker filter supports several further options that can be appended to its configuration to fine tune its behavior:

A colon-separated list of HTTP status codes that will trigger the fallback behavior. These can be expressed in either numerical or text format:
```
CircuitBreaker=myCircuitBreaker,forward:/inCaseOfFailureUseThis,401:NOT_FOUND:500
```
The percentage failure rate threshold above which the circuit breaker will be opened (default 50%, expressed as a float value):
```
CircuitBreaker=myCircuitBreaker,forward:/inCaseOfFailureUseThis,401:NOT_FOUND:500,10
```
The period of time in seconds to wait for downstream service recovery before entering the 'half-open' state and attempting to close the circuit breaker again (default 60s).
```
CircuitBreaker=myCircuitBreaker,forward:/inCaseOfFailureUseThis,401:NOT_FOUND:500,10,30
```

CORS in combination with Circuit Breaker The circuit breaker can be used in conjunction with CORS features, and CORS checks will be applied as expected to the inbound requests to circuit breaker protected routes. However, please note that in the event of the circuit breaker opening and forwarding requests to a fallback destination, the Origin header will be removed from these forwarded requests. This is required in order for the Gateway to perform this internal redirection.

Circuit breaker status

The status of a circuit breaker can be monitored by querying the metrics published on the Gateway's metrics actuator endpoint:

{gateway url}:8090/actuator/metrics/resilience4j.circuitbreaker.state?tag=state:{circuit-breaker-state}&tag=name:{circuit-breaker-name}

where {circuit-breaker-state} is one of open, closed, half_open, disabled (always allows access), forced_open (always denies access) or metrics_only, and {circuit-breaker-name} is the name of your circuit breaker, e.g. myCircuitBreaker

The metrics endpoint will return a value of 1 in the $.measurements[].value JSON path if the circuit breaker is in the given state.

For more information on these and other metrics, see Resilience4j CircuitBreaker Metrics.

`ClientCertificateHeader`

The ClientCertificateHeader filter validates the client TLS certificate used to make a request via the Gateway. You can also use this filter to validate the client certificate's fingerprint.

Note This filter relies on the Kubernetes container's ability to recognize a client certificate's Certificate Authority (CA).

To add client certificate validation to a route, add ClientCertificateHeader to the route's list of filters:

apiVersion: "tanzu.vmware.com/v1"
kind: SpringCloudGatewayRouteConfig
metadata:
  name: myapp-route-config
spec:
  service:
    name: myapp
  routes:
  - predicates:
      - Path=/api/**
    filters:
      - ClientCertificateHeader=*.example.com

To validate the client TLS certificate's fingerprint, append the type of the hash used for the fingerprint (either sha-1 or sha-256), and the expected fingerprint value, after the Common Name in the filter configuration.

As an example, the definition below uses the ClientCertificateHeader filter to ensure that received client certificates for requests to the /api/** route have a CN of *.example.com and a sha-1 fingerprint of aa:bb:00:99:

apiVersion: "tanzu.vmware.com/v1"
kind: SpringCloudGatewayRouteConfig
metadata:
  name: myapp-route-config
spec:
  service:
    name: myapp
  routes:
  - predicates:
      - Path=/api/**
    filters:
      - ClientCertificateHeader=*.example.com,sha-1:aa:bb:00:99

The fingerprint value is not case-sensitive, and the colon character : is not required to separate hexadecimal digits in a fingerprint. The following example works too:

apiVersion: "tanzu.vmware.com/v1"
kind: SpringCloudGatewayRouteConfig
metadata:
  name: myapp-route-config
spec:
  service:
    name: myapp
  routes:
  - predicates:
      - Path=/api/**
    filters:
      - ClientCertificateHeader=*.example.com,sha-1:AABB0099

`FallbackHeaders`

When a circuit breaker 'opens' and redirects a request to a fallback destination because the intended destination has returned an exception, it is sometimes useful for the fallback destination to receive information about the failure that just occurred. To facilitate this, the FallbackHeaders filter can be used to write the details of the exception into the headers of any requests forwarded to a fallback route.

The default header names added by the filter, and their values, are:

Execution-Exception-Type

The type (Java class) of the exception that tripped the circuit breaker
Execution-Exception-Message

The message attached to the exception that tripped the circuit breaker
Root-Cause-Exception-Type

The type (Java class) of the root cause exception, if available
Root-Cause-Exception-Message

The message attached to the root cause exception, if available

The example FallbackHeaders filter configuration below adds the exception information from failures in the /api/** route to the fallback requests made to the /inCaseOfFailureUseThis route:

apiVersion: "tanzu.vmware.com/v1"
kind: SpringCloudGatewayRouteConfig
metadata:
  name: myapp-route-config
spec:
  service:
    name: myapp
  routes:
    - predicates:
        - Path=/api/**
      filters:
        - CircuitBreaker="myCircuitBreaker,forward:/inCaseOfFailureUseThis"
    - uri: http://localhost:9994
      predicates:
        - Path=/inCaseOfFailureUseThis
      filters:
        - FallbackHeaders

You can optionally configure the names of the headers used by the filter, by appending a comma-separated list to the configuration. The alternative header names must be given in the same order as the default names listed above. So, the following configuration would update the Execution-Exception-Type header name to My-Execution-Exception-Type:

filters:
  - FallbackHeaders=My-Execution-Exception-Type

and the following configuration would update all of the default header names:

filters:
  - FallbackHeaders=My-Execution-Exception-Type,My-Execution-Exception-Message,My-Root-Cause-Exception-Type,My-Root-Cause-Exception-Message

`LocalResponseCache`

This filter adds a response cache at route level, reducing the number of requests sent from the gateway to upstream for a specific route. There is currently one implementation that uses in-memory caches: Caffeine project. Also, adding a global response cache to all the routes is possible.

According to HTTP specifications, you must consider the following when activating this feature:

It only caches responses when the request is bodiless and has the HTTP method GET.
It only caches responses with the following status codes: HTTP 200 (OK), HTTP 206 (Partial Content), and HTTP 301 (Moved Permanently).
Response data will not be cached if the Cache-Control header does not allow it (no-store in the request, no-store or private in the response).
If the matching cache response of a request is fresh but it contains the request Cache-Control directive no-cache, the cached response is ignored and a new fresh response coming from upstream is sent back to the client.
Currently, Spring Cloud Gateway doesn't implement re-validation, and therefore, it cannot send back the must-revalidate directive when the response is coming from cache.
When a response is coming from cache, the header Cache-Control will include the max-age directive
Spring Cloud Gateway responses usually have the Cache-Control directives must-revalidate, no-cache, and no-store, however, when the response is coming from cache they will not be present

The LocalResponseCache filter configuration accepts two parameters:

The timeToLive parameter sets the time allowed to elapse before a cache entry is evicted (expressed in s for seconds, m for minutes or h for hours)
The size parameter sets the maximum allowed size of the cache for this route before entries are evicted (expressed in KB, MB, or GB).

apiVersion: "tanzu.vmware.com/v1"
kind: SpringCloudGatewayRouteConfig
metadata:
  name: my-gateway-routes
spec:
  routes:
    - uri: https://httpbingo.org
      predicates:
        - Path=/get/**
      filters:
        - LocalResponseCache=3m,1MB

If neither of these parameters are specified, the default configuration as described in the global response cache guide will be used.

The LocalResponseCache filter will automatically update the max-age value in the Cache-Control header of returned responses, if max-age is present in the original response from the downstream service. If this is the case, then on the first call the max-age value will be rewritten with the number of seconds set in the timeToLive configuration parameter. In consecutive calls this value will be recalculated with the number of seconds left until the cached response expires.

Matching cache response of a request

Following the HTTP Specification about Cache-Control, Spring Cloud Gateway store a separate entry for each matching request. A single cache entry is identified by the combination of:

The URI, including request parameters
The Authentication header
The Cookies sent by the client
Any header specified by the Vary header

Examples:

GET http://gateway/api/1 and GET http://gateway/api/1 have the same cache entry
GET http://gateway/api/1 and GET http://gateway/api/2 have different entries
GET http://gateway/api/1?q=a and GET http://gateway/api/1?q=b have different entries
GET http://gateway/api/1 with Authorization: A and GET http://gateway/api/1 with Authorization: A have same entry
GET http://gateway/api/1 with Authorization: B and GET http://gateway/api/1 with Authorization: A have different entries
GET http://gateway/api/1 with Vary: X-My-Header + X-My-Header: A and GET http://gateway/api/1 with Vary: X-My-Header + X-My-Header: A have same entry
GET http://gateway/api/1 with Vary: X-My-Header + X-My-Header: B and GET http://gateway/api/1 with Vary: X-My-Header + X-My-Header: A have different entries

`RateLimit`

The RateLimit filter limits the number of requests allowed per route during a time window. It can apply rate limits to all requests to a route, or optionally per user or per source IP address.

In Gateway instances that have been configured to be highly available, where multiple Pods are serving traffic, rate limit request counts are synchronized across all the Pods that make up the Gateway instance. This ensures consistent rate limiting behavior regardless of the Gateway Pod which happens to serve a request. By default, this is done using an embedded Hazelcast distributed Map.

The RateLimit filter also supports Redis as an alternative shared store for rate limit data. When Redis is in use, the RateLimit filter will store all of its keys with the prefix scg:rate-limit:counters: to make them easily identifiable. To activate Redis and customize how data is stored, follow the same instructions as shown in the Using Redis as session storage guide.

Caution Activating Redis for the RateLimit filter implicitly activates the use of Redis for session storage. When updating a running instance, any request count or user session data previously held in Hazelcast will not be migrated.

Rate limiting is activated for a route by adding RateLimit to its list of filters. The RateLimit filter configuration accepts four parameters in the following order:

The maximum number of requests to be accepted during the window.
The duration of the window. By default this is specified in milliseconds, but you can use the s, m or h suffixes to specify it in seconds, minutes or hours respectively.
(Optional) User partition key. This allows different rate limits to be specified for different users, based on a user identifier to be found in the request. Set whether the key is to be found in a JWT claim or HTTP header with the {claim:CLAIM_NAME} or {header:HEADER_NAME} syntax.
(Optional) Source IP address, if the rate limit should be applied only to requests from a certain source. This cannot be combined with per user rate limits.

Basic rate limiting

The rate limit configuration shown below will perform basic limiting of the number requests allowed during a time window. In this example, a maximum of 1 request to the /api/** route will be allowed every 10 seconds:

apiVersion: "tanzu.vmware.com/v1"
kind: SpringCloudGatewayRouteConfig
metadata:
  name: myapp-route-config
spec:
  service:
    name: myapp
  routes:
  - predicates:
      - Path=/api/**
    filters:
      - RateLimit=1,10s

When requests are made at a rate allowed by the configured limit, the responses will succeed and report the remaining number of requests that can be made within the limit in the X-Remaining HTTP header. When the limit is exceeded, responses will fail with the 429 Too Many Requests status. The remaining time until requests will be allowed again is returned in the X-Retry-In HTTP header (specified in milliseconds).

Per-user rate limiting, partitioned by HTTP header

Below is an example configuration illustrating per-user rate limiting, where the user is identified from the contents of an HTTP header. In this example, an API key identifies the user, and is retrieved from the the X-API-Key header:

apiVersion: "tanzu.vmware.com/v1"
kind: SpringCloudGatewayRouteConfig
metadata:
  name: myapp-route-config
spec:
  service:
    name: myapp
  routes:
  - predicates:
      - Path=/api/**
    filters:
      - RateLimit=1,10s,{header:X-API-Key}

An individual rate limit of one request per ten seconds will be applied for each group of users sharing the same API key.

When requests for a given X-API-Key header exceed the defined limit in a time window, then access will be rejected with a simple 429 Too Many Requests response, with no additional headers.

Per-user rate limiting, partitioned by JWT claim

Similarly, the example configuration below illustrates per-user rate limiting again, this time with the user being identified by the value of the client-id claim of the JWT authorization token associated with the request:

apiVersion: "tanzu.vmware.com/v1"
kind: SpringCloudGatewayRouteConfig
metadata:
  name: myapp-route-config
spec:
  service:
    name: myapp
  routes:
  - predicates:
      - Path=/api/**
    filters:
      - RateLimit=1,10s,{claim:client-id}

When requests for a given client-id value exceed the defined limit in a time window, then access will be rejected with a simple 429 Too Many Requests response, with no additional headers.

Limiting by IP address

To limit by IP address, the RateLimit filter configuration accepts a semi-colon separated list of source IP addresses from which to accept requests.

The filter checks the X-Forwarded-For header, if present, for the source IP address of the request. Since multiple IP addresses may have been added to the X-Forwarded-For header by multiple trusted or untrusted proxies, you can optionally set the maximum trusted index of the IP to read from the header, by prepending an integer value to the IP address list. The default value of 1 will read the last IP from the header, while a value of 2 will read up to the second from last IP, and so on. The index value must be greater than zero.

Important If you are using an Ingress, ensure it is configured to pass the incoming X-Forwarded-For header downstream to the Gateway.

Here is an example configuration to rate limit by IP address:

apiVersion: "tanzu.vmware.com/v1"
kind: SpringCloudGatewayRouteConfig
metadata:
  name: myapp-route-config
spec:
  service:
    name: myapp
  routes:
  - predicates:
      - Path=/api/**
    filters:
      - RateLimit=1,10s,{IPs:2;127.0.0.1;192.168.0.1}

In this example, a maximum of 1 request is allowed in a 10s window, as long as it originates from any of the configured IPs: 127.0.0.1 and 192.168.0.1.

The maximum trusted index to read from the X-Forwarded-For header is set to 2. If, for example, the X-Forwarded-For header had a value of 4.4.4.4, 8.8.8.8, 127.0.0.1, the gateway would return 403 Forbidden because the second-last IP, 8.8.8.8, is not in the allowed list. However, if the header was instead set to 4.4.4.4, 127.0.0.1, 8.8.8.8, then the gateway would return successfully, because the second-last IP would then be 127.0.0.1, and this is in the allowed list.

`RestrictRequestHeaders`

This filter controls the HTTP headers that are allowed on a request. If there are any HTTP headers that are not in the header list configuration (case insensitive) then a response of 403 Forbidden error will be returned to client.

In this example, only requests with "Content-Type" and "X-Request-Temp" headers will be accepted:

apiVersion: "tanzu.vmware.com/v1"
kind: SpringCloudGatewayRouteConfig
metadata:
  name: my-gateway-routes
spec:
  service:
    name: myapp
  routes:
  - ssoEnabled: true
    predicates:
      - Path=/api/**
    filters:
      - RestrictRequestHeaders=Content-Type,x-request-temp

Note: If any load balancers or network gateways add extra request headers, they need to be included in the list or the request will return an error.