Commercial route filters for traffic control for SCG for K8s

The open-source Spring Cloud Gateway project includes a number of built-in filters for use in Gateway routes. The following commercial filters provided by VMware Spring Cloud Gateway for Kubernetes can be used in addition to those included in the OSS project.

`AllowedRequestCookieCount`

The AllowedRequestCookieCount filter controls the maximum number of cookies allowed on a request. The filter responds with a "431 Request Header Fields Too Large" error if the number of cookies on a request exceeds the configured integer maximum.

In this example, only requests with two or fewer cookies are accepted:

apiVersion: "tanzu.vmware.com/v1"
kind: SpringCloudGatewayRouteConfig
metadata:
  name: my-gateway-routes
spec:
  service:
    name: myapp
  routes:
    - ssoEnabled: true
      predicates:
        - Path=/api/**
      filters:
        - AllowedRequestCookieCount=2

`AllowedRequestHeadersCount`

The AllowedRequestHeadersCount filter controls the maximum number of headers allowed on a request. The filter responds with a "431 Request Header Fields Too Large" error if the number of headers on a request exceeds the configured integer maximum.

In this example, only requests with four or fewer headers are accepted:

apiVersion: "tanzu.vmware.com/v1"
kind: SpringCloudGatewayRouteConfig
metadata:
  name: my-gateway-routes
spec:
  service:
    name: myapp
  routes:
  - ssoEnabled: true
    predicates:
      - Path=/api/**
    filters:
      - AllowedRequestHeadersCount=4

`AllowedRequestQueryParamsCount`

The AllowedRequestQueryParamsCount filter controls the maximum number of query parameters allowed on a request. The filter will respond with a "414 URL Too Large" error if the number of query parameters on a request exceeds the configured integer maximum.

In this example, only requests with three or fewer query parameters will be accepted:

apiVersion: "tanzu.vmware.com/v1"
kind: SpringCloudGatewayRouteConfig
metadata:
  name: my-gateway-routes
spec:
  service:
    name: myapp
  routes:
    - ssoEnabled: true
      predicates:
        - Path=/api/**
      filters:
        - AllowedRequestQueryParamsCount=3

`CircuitBreaker`

The CircuitBreaker filter reroutes requests to a fallback destination when the intended destination responds with an HTTP error code.

The circuit breaker pattern is a useful approach for bolstering resilience and preventing cascading failures in distributed systems. It works by preventing struggling services from becoming overloaded with failing requests, giving them chance to recover.

In electrical systems, a circuit breaker is a switch that triggers on a detected fault, isolating the faulty circuit and preventing further damage. This act is known as opening the circuit breaker (breaking the circuit). When the fault is corrected, the circuit breaker is closed (completing the circuit), allowing current to flow again. The circuit breaker pattern borrows this highly useful technique and terminology and applies it to distributed software systems.

Because our systems must be able to recover autonomously, there is also a half-open state which allows the circuit breaker to detect recovery of the downstream system. Following a timeout after the original detected failure, the circuit breaker allows trial requests through to the destination system, and then either fully closes, or fully opens and restarts the timeout, based on the status of the response.

In the following example, the CircuitBreaker filter is used to cause requests to /api/** to fall back to /inCaseOfFailureUseThis, should the myapp service respond with an error code:

apiVersion: "tanzu.vmware.com/v1"
kind: SpringCloudGatewayRouteConfig
metadata:
  name: myapp-route-config
spec:
  service:
    name: myapp
  routes:
  - predicates:
      - Path=/api/**
    filters:
      - CircuitBreaker=myCircuitBreaker,forward:/inCaseOfFailureUseThis

The CircuitBreaker filter supports several options that can be appended to its configuration to fine tune its behavior:

A colon-separated list of HTTP status codes that trigger the fallback behavior. These can be expressed in either numerical or text format:
```
CircuitBreaker=myCircuitBreaker,forward:/inCaseOfFailureUseThis,401:NOT_FOUND:500
```
The percentage failure rate threshold above which the circuit breaker is opened (default 50%, expressed as a float value):
```
CircuitBreaker=myCircuitBreaker,forward:/inCaseOfFailureUseThis,401:NOT_FOUND:500,10
```
The period of time in seconds to wait for downstream service recovery before entering the "half-open" state and attempting to close the circuit breaker again (default 60s).
```
CircuitBreaker=myCircuitBreaker,forward:/inCaseOfFailureUseThis,401:NOT_FOUND:500,10,30
```

CORS in combination with Circuit Breaker The circuit breaker can be used in conjunction with CORS features, and CORS checks will be applied as expected to the inbound requests to circuit breaker protected routes. However, please note that in the event of the circuit breaker opening and forwarding requests to a fallback destination, the Origin header will be removed from these forwarded requests. This is required in order for the Gateway to perform this internal redirection.

Circuit breaker status

The status of a circuit breaker can be monitored by querying the metrics published on the Gateway's metrics actuator endpoint:

{gateway url}:8090/actuator/metrics/resilience4j.circuitbreaker.state?tag=state:{circuit-breaker-state}&tag=name:{circuit-breaker-name}

Where:

{circuit-breaker-state} is one of:
- open
- closed
- half_open
- disabled (always allows access)
- forced_open (always denies access)
- metrics_only
{circuit-breaker-name} is the name of your circuit breaker, for example, myCircuitBreaker

The metrics endpoint returns a value of 1 in the $.measurements[].value JSON path if the circuit breaker is in the given state.

For more information about these and other metrics, see Resilience4j CircuitBreaker Metrics.

`ClientCertificateHeader`

The ClientCertificateHeader filter validates the client TLS certificate used to make a request using the Gateway. You can also use this filter to validate the client certificate's fingerprint.

Note This filter relies on the Kubernetes container's ability to recognize a client certificate's Certificate Authority (CA).

To add client certificate validation to a route, add ClientCertificateHeader to the route's list of filters:

apiVersion: "tanzu.vmware.com/v1"
kind: SpringCloudGatewayRouteConfig
metadata:
  name: myapp-route-config
spec:
  service:
    name: myapp
  routes:
  - predicates:
      - Path=/api/**
    filters:
      - ClientCertificateHeader=*.example.com

To validate the client TLS certificate's fingerprint, append the type of the hash used for the fingerprint (either sha-1 or sha-256), and the expected fingerprint value, after the Common Name in the filter configuration.

As an example, the following definition uses the ClientCertificateHeader filter to ensure that received client certificates for requests to the /api/** route have a CN of *.example.com and a sha-1 fingerprint of aa:bb:00:99:

apiVersion: "tanzu.vmware.com/v1"
kind: SpringCloudGatewayRouteConfig
metadata:
  name: myapp-route-config
spec:
  service:
    name: myapp
  routes:
  - predicates:
      - Path=/api/**
    filters:
      - ClientCertificateHeader=*.example.com,sha-1:aa:bb:00:99

The fingerprint value is not case-sensitive, and the colon character : is not required to separate hexadecimal digits in a fingerprint. The following example works, too:

apiVersion: "tanzu.vmware.com/v1"
kind: SpringCloudGatewayRouteConfig
metadata:
  name: myapp-route-config
spec:
  service:
    name: myapp
  routes:
  - predicates:
      - Path=/api/**
    filters:
      - ClientCertificateHeader=*.example.com,sha-1:AABB0099

`FallbackHeaders`

When a circuit breaker "opens" and redirects a request to a fallback destination because the intended destination has returned an exception, it is sometimes useful for the fallback destination to receive information about the failure that just occurred. To facilitate this, the FallbackHeaders filter can be used to write the details of the exception into the headers of any requests forwarded to a fallback route.

The default header names added by the filter, and their values, are:

Execution-Exception-Type

The type (Java class) of the exception that tripped the circuit breaker
Execution-Exception-Message

The message attached to the exception that tripped the circuit breaker
Root-Cause-Exception-Type

The type (Java class) of the root cause exception, if available
Root-Cause-Exception-Message

The message attached to the root cause exception, if available

The following example FallbackHeaders filter configuration adds the exception information from failures in the /api/** route to the fallback requests made to the /inCaseOfFailureUseThis route:

apiVersion: "tanzu.vmware.com/v1"
kind: SpringCloudGatewayRouteConfig
metadata:
  name: myapp-route-config
spec:
  service:
    name: myapp
  routes:
    - predicates:
        - Path=/api/**
      filters:
        - CircuitBreaker="myCircuitBreaker,forward:/inCaseOfFailureUseThis"
    - uri: http://localhost:9994
      predicates:
        - Path=/inCaseOfFailureUseThis
      filters:
        - FallbackHeaders

You can, if desired, configure the names of the headers used by the filter by appending a comma-separated list to the configuration. The alternative header names must be given in the same order as the default names listed above. So the following configuration updates the Execution-Exception-Type header name to My-Execution-Exception-Type:

filters:
  - FallbackHeaders=My-Execution-Exception-Type

And the following configuration updates all of the default header names:

filters:
  - FallbackHeaders=My-Execution-Exception-Type,My-Execution-Exception-Message,My-Root-Cause-Exception-Type,My-Root-Cause-Exception-Message

`LocalResponseCache`

The LocalResponseCache filter adds a response cache at route level, reducing the number of requests sent from the gateway to upstream for a specific route. There is currently one implementation that uses in-memory caches: Caffeine project. Also, it's possible to add a global response cache to all the routes.

According to HTTP specifications, you must consider the following when activating this feature:

It only caches responses when the request is bodiless and has the HTTP method GET.
It only caches responses with the following status codes:
- HTTP 200 (OK)
- HTTP 206 (Partial Content)
- HTTP 301 (Moved Permanently)
Response data will not be cached if the Cache-Control header does not allow it (no-store in the request, no-store or private in the response).
If the matching cache response of a request is fresh, but it contains the request Cache-Control directive no-cache, the cached response is ignored and a new fresh response coming from upstream is sent back to the client.
Currently, Spring Cloud Gateway doesn't implement re-validation, so it cannot send back the must-revalidate directive when the response is coming from cache.
When a response is coming from cache, the header Cache-Control includes the max-age directive.
Spring Cloud Gateway responses usually have the Cache-Control directives must-revalidate, no-cache, and no-store, however, when the response is coming from cache they will not be present.

The LocalResponseCache filter configuration accepts two parameters:

The timeToLive parameter sets the time allowed to elapse before a cache entry is evicted (expressed in s for seconds, m for minutes or h for hours)
The size parameter sets the maximum allowed size of the cache for this route before entries are evicted (expressed in KB, MB, or GB).
```
apiVersion: "tanzu.vmware.com/v1"
kind: SpringCloudGatewayRouteConfig
metadata:
  name: my-gateway-routes
spec:
  routes:
    - uri: https://httpbingo.org
      predicates:
        - Path=/get/**
      filters:
        - LocalResponseCache=3m,1MB
```
If neither of these parameters is specified, the default configuration as described in the global response cache guide is used.

The LocalResponseCache filter automatically updates the max-age value in the Cache-Control header of returned responses, if max-age is present in the original response from the downstream service. If this is the case, then on the first call, the max-age value is rewritten with the number of seconds set in the timeToLive configuration parameter. In consecutive calls, this value is recalculated with the number of seconds remaining until the cached response expires.

Matching cache response of a request

Following the HTTP Cache-Control specification, Spring Cloud Gateway stores a separate entry for each matching request. A single cache entry is identified by the combination of:

The URI, including request parameters
The Authentication header
The cookies sent by the client
Any header specified by the Vary header

Examples:

Request 1	Request 2	same/different cache entry
GET http://gateway/api/1	GET http://gateway/api/1	same
GET http://gateway/api/1	GET http://gateway/api/2	different
GET http://gateway/api/1?q=a	GET http://gateway/api/1?q=b	different
GET http://gateway/api/1 with `Authorization: A`	GET http://gateway/api/1 with `Authorization: A`	same
GET http://gateway/api/1 with `Authorization: B`	GET http://gateway/api/1 with `Authorization: A`	different
GET http://gateway/api/1 with `Vary: X-My-Header` + `X-My-Header: A`	GET http://gateway/api/1 with `Vary: X-My-Header` + `X-My-Header: A`	same
GET http://gateway/api/1 with `Vary: X-My-Header` + `X-My-Header: B`	GET http://gateway/api/1 with `Vary: X-My-Header` + `X-My-Header: A`	different

`RateLimit`

The RateLimit filter limits the number of requests allowed per route during a time window. It can apply rate limits to:

all requests to a route
per user
per source IP address

In Gateway instances that have been configured to be highly available, where multiple Pods are serving traffic, rate limit request counts are synchronized across all the Pods that make up the Gateway instance. This ensures consistent rate limiting behavior, regardless of the Gateway Pod that happens to serve a request. By default, this is done using an embedded Hazelcast distributed Map.

The RateLimit filter also supports Redis as an alternative shared store for rate limit data. When Redis is in use, the RateLimit filter stores all of its keys with the prefix scg:rate-limit:counters: to make them easily identifiable. To activate Redis and customize how data is stored, follow the instructions in Using Redis as session storage.

Caution Activating Redis for the RateLimit filter implicitly activates the use of Redis for session storage. When updating a running instance, any request count or user session data previously held in Hazelcast will not be migrated.

Rate limiting is activated for a route by adding RateLimit to its list of filters. The RateLimit filter configuration accepts four parameters in the following order:

The maximum number of requests to be accepted during the window.
The duration of the window. By default this is specified in milliseconds, but you can use the s, m, or h suffixes to specify it in seconds, minutes, or hours respectively.
One of:
1. (Optional) User partition key. This allows different rate limits to be specified for different users, based on a user identifier found in the request. Set whether the key is to be found in a JWT claim or HTTP header with the {claim:CLAIM_NAME} or {header:HEADER_NAME} syntax.
2. (Optional) Source IP address, if the rate limit should be applied only to requests from a certain source. This cannot be combined with per user rate limits.

Basic rate limiting

The following rate limit configuration performs basic limiting of the number requests allowed during a time window. In this example, a maximum of 1 request to the /api/** route is allowed every 10 seconds:

apiVersion: "tanzu.vmware.com/v1"
kind: SpringCloudGatewayRouteConfig
metadata:
  name: myapp-route-config
spec:
  service:
    name: myapp
  routes:
  - predicates:
      - Path=/api/**
    filters:
      - RateLimit=1,10s

When requests are made at a rate allowed by the configured limit, the responses succeed and they report the remaining number of requests that can be made within the limit in the X-Remaining HTTP header. When the limit is exceeded, responses fail with the "429 Too Many Requests" status. The remaining time until requests are allowed again is returned in the X-Retry-In HTTP header (specified in milliseconds).

Per-user rate limiting, partitioned by HTTP header

The following is an example configuration illustrating per-user rate limiting, where the user is identified from the contents of an HTTP header. In this example, an API key identifies the user, and is retrieved from the the X-API-Key header:

apiVersion: "tanzu.vmware.com/v1"
kind: SpringCloudGatewayRouteConfig
metadata:
  name: myapp-route-config
spec:
  service:
    name: myapp
  routes:
  - predicates:
      - Path=/api/**
    filters:
      - RateLimit=1,10s,{header:X-API-Key}

An individual rate limit of one request per ten seconds is applied for each group of users sharing the same API key.

When requests for a given X-API-Key header exceed the defined limit in a time window, then access will be rejected with the "429 Too Many Requests" response, with no additional headers.

Per-user rate limiting, partitioned by JWT claim

Similarly, the following example configuration illustrates per-user rate limiting, this time with the user being identified by the value of the client-id claim of the JWT authorization token associated with the request:

apiVersion: "tanzu.vmware.com/v1"
kind: SpringCloudGatewayRouteConfig
metadata:
  name: myapp-route-config
spec:
  service:
    name: myapp
  routes:
  - predicates:
      - Path=/api/**
    filters:
      - RateLimit=1,10s,{claim:client-id}

When requests for a given client-id value exceed the defined limit in a time window, then access is rejected with the "429 Too Many Requests" response, with no additional headers.

Limiting by IP address

To limit by IP address, the RateLimit filter configuration accepts a semi-colon separated list of source IP addresses from which to accept requests.

The filter checks the X-Forwarded-For header, if present, for the source IP address of the request. Since multiple IP addresses may have been added to the X-Forwarded-For header by multiple trusted or untrusted proxies, you can optionally set the maximum trusted index of the IP to read from the header by prepending an integer value to the IP address list. The default value of 1 will read the last IP from the header, while a value of 2 will read up to the second from last IP, and so on. The index value must be greater than zero.

Important If you are using an Ingress, ensure that it is configured to pass the incoming X-Forwarded-For header downstream to the Gateway.

Here is an example configuration to rate limit by IP address:

apiVersion: "tanzu.vmware.com/v1"
kind: SpringCloudGatewayRouteConfig
metadata:
  name: myapp-route-config
spec:
  service:
    name: myapp
  routes:
  - predicates:
      - Path=/api/**
    filters:
      - RateLimit=1,10s,{IPs:2;127.0.0.1;192.168.0.1}

In this example, a maximum of 1 request is allowed in a 10s window, as long as it originates from any of the configured IPs: 127.0.0.1 and 192.168.0.1.

The maximum trusted index to read from the X-Forwarded-For header is set to 2. If, for example, the X-Forwarded-For header has a value of 4.4.4.4, 8.8.8.8, 127.0.0.1, the gateway returns "403 Forbidden" because the second-last IP, 8.8.8.8, is not in the allowed list. However, if the header is instead set to 4.4.4.4, 127.0.0.1, 8.8.8.8, then the gateway returns successfully, because the second-last IP is then 127.0.0.1, and this is in the allowed list.

`RestrictRequestHeaders`

The RestrictRequestHeaders filter controls the HTTP headers that are allowed on a request. If there are any HTTP headers that are not in the header list configuration (case insensitive), then a response of "403 Forbidden" is returned to client.

In this example, only requests with "Content-Type" and "X-Request-Temp" headers are accepted:

apiVersion: "tanzu.vmware.com/v1"
kind: SpringCloudGatewayRouteConfig
metadata:
  name: my-gateway-routes
spec:
  service:
    name: myapp
  routes:
  - ssoEnabled: true
    predicates:
      - Path=/api/**
    filters:
      - RestrictRequestHeaders=Content-Type,x-request-temp

Note If any load balancers or network gateways add extra request headers, they must be included in the list or the request will return an error.