This topic describes how to define network profile to configure the NSX-T Load Balancer for VMware Tanzu Kubernetes Grid Integrated Edition.
The NSX-T Load Balancer is a logical load balancer that handles a number of functions using virtual servers and pools.
The NSX-T load balancer creates a load balancer service for each Kubernetes cluster provisioned by Tanzu Kubernetes Grid Integrated Edition with NSX-T. For each load balancer service, NCP, by way of the CRD, creates corresponding NSXLoadBalancerMonitor objects.
By default Tanzu Kubernetes Grid Integrated Edition deploys the following NSX-T virtual servers for each Kubernetes cluster:
type: LoadBalancer
.For information about configuring TCP layer 4 ingress controller see Configure the TCP Ingress Controller Network Profile, below.
For information about configuring layer 7 ingress routing load balancers see Scaling the HTTP/S Layer 7 Ingress Load Balancers Using the LoadBalancer CRD. For information about configuring the layer 7 ingress controller see Defining Network Profiles for the HTTP/S Layer 7 Ingress Controller.
For more information about the NSX-T Load Balancer, see Create an IP Pool in Manager Mode or Add an IP Address Pool in the VMware documentation.
For more information about Kubernetes custom resources, see Custom resources in the Kubernetes documentation.
The TCP layer 4 virtual server provisioned for each Kubernetes service is controlled by the parameters exposed in a network profile.
Note: The TCP layer 4 virtual server that fronts the Kubernetes API server is always created, and it is not controlled by the parameters exposed in the network profile.
The NSX Ingress Controller is configured using the ncp.ini
network profile configuration file.
The TCP Ingress Controller network profile has the following format:
{
"name": "network_profile",
"description": "DESCRIP",
"parameters" : {
"cni_configurations": {
"type": "nsxt",
"parameters": {
"nsx_lb": NSX-LB,
"x_forwarded_for": "FORWARD-TYPE",
"max_l4_lb_service": MAX-SERVERS,
"l4_persistence_type": "source_ip",
"l4_lb_algorithm": "LB-BEHAVIOR"
}
}
}
}
Where:
DESCRIP
is your description for this network profile configuration.NSX-LB
is your preference for whether the NSX-T Load Balancer is used for your Kubernetes clusters. For more information see Configure Which NSX Load Balancer to Use, below.FORWARD-TYPE
(Optional) is your request header original client source IP. For more information see the Configure the Client Source IP Address, below.MAX-SERVERS
is your maximum number of layer 4 virtual servers per cluster. For more information see the Configure the Maximum Number of Layer 4 Load Balancer Virtual Servers, below.LB-BEHAVIOR
(Optional) is your load balancer behavior. For more information see the Configure the Layer 4 Load Balancer Algorithm, below.{
"name": "network_profile",
"description": "DESCRIP",
"parameters" : {
"cni_configurations": {
"type": "nsxt",
"parameters": {
"nsx_lb": true,
"x_forwarded_for": "replace",
"max_l4_lb_service": 10,
"l4_persistence_type": "source_ip",
"l4_lb_algorithm": "weighted_round_robin"
}
}
}
}
The following table describes the Ingress Controller configuration parameters:
Parameter | Type | Description |
---|---|---|
name |
String | User-defined name of the network profile. |
description |
String | User-defined description for the network profile. |
parameters |
Map | One or more name-value pairs. |
cni_configurations |
Map | Map containing two key-value pairs: type and parameters . |
type |
Constant String | Only nsxt is accepted. |
parameters |
Map | Map containing one or more key-value pairs for NCP settings. |
nsx_lb |
Boolean | Flag to control if the NSX-T Load Balancer is used for the Kubernetes cluster. |
x_forwarded_for |
String | Sets the original client source IP in the request header. Default is none. Accepts “insert” and “replace” .Enabling the network profile x_forwarded_for parameter automatically enables the x_forwarded_port and x_forward_protocol parameters. |
max_l4_lb_service |
Integer | Limit the maximum number of layer 4 virtual servers per cluster. Minimum is 1 . |
l4_persistence_type |
String | Connection stickiness based on source_ip (only accepted value). |
l4_lb_algorithm |
String | Specify the layer 4 load balancer behavior. Accepts “round_robin” (default), “least_connection” , “ip_hash” , “weighted_round_robin” . |
The nsx_lb
parameter is used to control the TCP layer 4 virtual server that is provisioned for each Kubernetes service of type: LoadBalancer.
The nsx_ingress_controller
parameter is used to control if NCP is used as the Ingress Controller for the Kubernetes cluster. By default when you define an ingress resource for a Kubernetes cluster, NCP instructs the NSX-T load balancer to provision 2 layer 7 virtual services (HTTP and HTTPS) as the Ingress Controller.
The nsx_ingress_controller
parameter is subject to the nsx_lb
parameter as described in the following table:
nsx_lb setting | nsx_ingress_controller: true |
nsx_ingress_controller: false |
---|---|---|
nsx_lb: true |
Use the NSX-T Layer 4 LoadBalancer and the NCP-provisioned Layer 7 Ingress Controller. | Use the NSX-T Layer 4 LoadBalancer and a third-party Ingress Controller, such as NGINX. |
nsx_lb: false |
Invalid configuration. You cannot deactivate the NSX-T Load Balancer and use NCP as the Ingress Controller. The network profile will fail validation. | Use a third-party load balancer and a third-party ingress controller. |
The nsx_lb
flag controls whether to deploy either the NSX-T Load Balancer or a third-party load balancer, such as Nginx.
The nsx_lb
parameter accepts true
or false
. The default setting is true
and the NSX-T Load Balancer is deployed. To use a third party load balancer, set this parameter to false
.
For example:
{
"name": "example_network_profile",
"description": "nsx_lb and nsx_ingress_controller are enabled",
"parameters": {
"cni_configurations": {
"type": "nsxt",
"parameters": {
"nsx_lb": false,
"nsx_ingress_controller": false
}
}
}
}
Note: The nsx_lb
parameter maps to the use_native_loadbalancer
parameter in NCP.ini.
Since the nsx_ingress_controller
is a component of the NSX-T Load Balancer, if you choose to use a third-party load balancer (nsx_lb: false
), you must also explicitly deactivate the nsx_ingress_controller
. See Defining Network Profiles for the HTTP/S Layer 7 Ingress Controller.
The X-Forwarded-For HTTP header field is used to identify the originating IP address of a client connecting to a web server through an HTTP proxy or load balancer.
In network profile, the x_forwarded_for
parameter can be enabled to ensure that the client IP address will be set in the HTTP request header. The x_forwarded_for
parameter is useful in situations where it is important to know the source IP address of the client request, such as for auditing purposes.
Note: Enabling the network profile x_forwarded_for
parameter automatically enables the x_forwarded_port
and x_forward_protocol
parameters. This is supported with NSX-T v3.0.x and later.
The x_forwarded_for
parameter type is String that accepts "insert"
and "replace"
. Any other type will be rejected. Missing entry is accepted.
If x_forwarded_for
is set to “insert”, the client source IP will be appended (comma separated) to the existing set of client source IP addresses. If x_forwarded_for
is set to “replace”, any existing client source IP address will be replaced with the current client source IP address.
For example, with the following network profile the source IP address of the client will be appended to the existing set of client source IP addresses:
{
"name": "example-network-profile",
"description": "x_forwarded_for insert",
"parameters" : {
"cni_configurations": {
"type": "nsxt",
"parameters": {
"x_forwarded_for": "insert"
}
}
}
}
For example, with the following network profile the existing source IP address of the client will replace all other client source IP entries:
{
"name": "example-network-profile",
"description": "x_forwarded_for replace",
"parameters" : {
"cni_configurations": {
"type": "nsxt",
"parameters": {
"x_forwarded_for": "replace"
}
}
}
}
The default NSX-T Load Balancer behavior is that auto-scaling is unlimited. This means that the number of layer 4 virtual servers that can be deployed is governed only by the capacity of the Edge Cluster where the load balancer service is deployed. As a result, in theory it is possible for a single Kubernetes cluster to use up all of the layer 4 virtual servers that the Edge Cluster can support.
The max_l4_service
parameter sets the upper limit for the number of virtual servers that can be used by a Kubernetes cluster. You can use this parameter to limit the number of virtual servers that can be created per Kubernetes cluster.
The max_l4_lb_service
data type is an integer. The value must be larger or equal to 1. Missing entry is accepted.
For example, the following network profile uses the max_l4_lb_service
parameter to limit the number of layer 4 virtual servers to 100 per cluster:
{
"name": "example_network_profile",
"description": "max_l4_lb_service",
"parameters" : {
"cni_configurations": {
"type": "nsxt",
"parameters": {
"max_l4_lb_service": 100
}
}
}
}
The l4_persistence_type
is used to set connection stickiness based on source_ip
.
The l4_persistence_type
data type is string. The only accepted value is source_ip
.
{
"name": "example_network_profile",
"description": "l4_persistence_type",
"parameters": {
"cni_configurations": {
"type": "nsxt",
"parameters": {
"l4_persistence_type": "source_ip"
}
}
}
}
The l4_lb_algorithm
is used to set the algorithm type for the layer 4 NSX-T Load Balancer service.
The l4_lb_algorithm
data type is string enumeration that accepts one of the following values:
"round_robin"
(default)"least_connection"
"ip_hash"
"weighted_round_robin"
For example, the following network profile specifies the weighted_round_robin
as the load balancer algorithm:
{
"name": "example_network_profile",
"description": "l4_lb_algorithm",
"parameters": {
"cni_configurations": {
"type": "nsxt",
"parameters": {
"l4_lb_algorithm": "weighted_round_robin"
}
}
}
}