When there are too many API requests sent at a time, it affects the performance of the system. You can enable Rate Limiting, which enforces a limit on the number of API requests sent by each user.

The SASE Orchestrator makes use of certain defence mechanisms that curb API abuse and provides system stability. API requests that exceed the allowed request limits are blocked and returned with HTTP 429 (Too many Requests). The system needs to go through a cool down period before making the requests again.

The following types of Rate-Limiters are deployed on SASE Orchestrator:
  • Leaky bucket limiter – Smooths the burst of requests and only allows a pre-defined number of requests. This limiter takes care of limiting the number of requests allowed in a given time window.
  • Concurrency limiter – Limits the number of requests that occur in parallel which leads to concurrent requests fighting for resources and may result in long running queries.
The following are the major reasons that lead to rate limiting of the API requests:
  • Large number of active or concurrent requests.
  • Sudden spikes in request volume.
  • Requests resulting in long running queries on the Orchestrator holding system resources for long being dropped.
Developers that rely on the API can adopt the following measures to improve the stability of their code when the VCO rate-limiting capability is enabled.
  • Handle HTTP 429 response code when requests exceed rate limits.
  • The penalty time duration is 5000 ms when the rate limiter reaches the maximum allowed requests in a given period. If blocked, the clients are expected to have a cool down period of 5000 ms before making requests again. The requests made during the cool down period of 5000 ms will still be rate limited.
  • Use shorter time intervals for time series APIs which will not let the request to expire due to long running queries.
  • Prefer batch query methods to those that query individual Customers or Edges whenever possible.
Note: Operator Super users configure Rate limits discretely based on the environment. For any queries on relevant policies, contact your Operator.

Configure Rate Limiting Policies using System Properties

You can use the following system properties to enable Rate Limiting and define the default set of policies:
  • vco.api.rateLimit.enabled
  • vco.api.rateLimit.mode.logOnly
  • vco.api.rateLimit.rules.global
  • vco.api.rateLimit.rules.enterprise.default
  • vco.api.rateLimit.rules.enterpriseProxy.default

For more information on the system properties, see List of System Properties.

Configure Rate Limiting Policies using APIs

It is recommended to configure the rate limiter policies as global rules using the system properties, as this approach produces the best possible API performance, facilitates troubleshooting, and ensures a consistent user experience across all Partners and Customers. In rare cases, however, Operators may determine that global policies are too lax for a particular tenant or user. For such cases, VMware supports the following operator-only APIs to set policies for specific partners and enterprises.
  • enterpriseProxy/insertOrUpdateEnterpriseProxyRateLimits – Used to configure Partner-specific policies.
  • enterprise/insertOrUpdateEnterpriseRateLimits – Used to configure Customer-specific policies.

For more information on the APIs, see https://code.vmware.com/apis/1037/velocloud-sdwan-vco-api.