This topic describes how to define network profile to configure the NSX Load Balancer for VMware Tanzu Kubernetes Grid Integrated Edition (TKGI) provisioned Kubernetes clusters.
The NSX-T Load Balancer is a logical load balancer that handles a number of functions using virtual servers and pools.
The NSX-T load balancer creates a load balancer service for each Kubernetes cluster provisioned by Tanzu Kubernetes Grid Integrated Edition with NSX-T. For each load balancer service, NCP, by way of the Kubernetes CustomResourceDefinition (CRD), creates corresponding NSXLoadBalancerMonitor objects.
By default Tanzu Kubernetes Grid Integrated Edition deploys the following NSX-T virtual servers for each Kubernetes cluster:
type: LoadBalancer
.For information about configuring TCP layer 4 ingress controller see Configure the TCP Ingress Controller Network Profile, below.
For information about configuring layer 7 ingress routing load balancers see Scaling the HTTP/S Layer 7 Ingress Load Balancers Using the LoadBalancer CRD. For information about configuring the layer 7 ingress controller see Defining Network Profiles for the HTTP/S Layer 7 Ingress Controller.
For more information about the NSX-T Load Balancer, see Create an IP Pool in Manager Mode or Add an IP Address Pool in the VMware documentation.
For more information about Kubernetes custom resources, see Custom resources in the Kubernetes documentation.
The TCP layer 4 virtual server provisioned for each Kubernetes service is controlled by the parameters exposed in a network profile.
Note: The TCP layer 4 virtual server that fronts the Kubernetes API server is always created, and it is not controlled by the parameters exposed in the network profile.
The NSX Ingress Controller is configured using the ncp.ini
network profile configuration file.
The TCP Ingress Controller network profile has the following format:
{
"name": "network_profile",
"description": "DESCRIP",
"parameters" : {
"cni_configurations": {
"type": "nsxt",
"parameters": {
"nsx_lb": NSX-LB,
"x_forwarded_for": "FORWARD-TYPE",
"max_l4_lb_service": MAX-SERVERS,
"l4_persistence_type": "source_ip",
"l4_lb_algorithm": "LB-BEHAVIOR"
}
}
}
}
Where:
DESCRIP
is your description for this network profile configuration.NSX-LB
is your preference for whether the NSX-T Load Balancer is used for your Kubernetes clusters. For more information see Configure Which NSX Load Balancer to Use, below.FORWARD-TYPE
(Optional) is your request header original client source IP. For more information see the Configure the Client Source IP Address, below.MAX-SERVERS
is your maximum number of layer 4 virtual servers per cluster. For more information see the Configure the Maximum Number of Layer 4 Load Balancer Virtual Servers, below.LB-BEHAVIOR
(Optional) is your load balancer behavior. For more information see the Configure the Layer 4 Load Balancer Algorithm, below.{
"name": "network_profile",
"description": "DESCRIP",
"parameters" : {
"cni_configurations": {
"type": "nsxt",
"parameters": {
"nsx_lb": true,
"x_forwarded_for": "replace",
"max_l4_lb_service": 10,
"l4_persistence_type": "source_ip",
"l4_lb_algorithm": "weighted_round_robin"
}
}
}
}
The following table describes the Ingress Controller configuration parameters:
Parameter | Type | Description |
---|---|---|
name |
String | User-defined name of the network profile. |
description |
String | User-defined description for the network profile. |
parameters |
Map | Map containing one or more key-value pairs. |
cni_configurations |
Map | Map containing type and parameters key-value pairs for configuring NCP. |
type |
Constant String |
Values: “nsxt” . |
parameters |
Map | Map containing one or more key-value pairs for NCP settings. |
nsx_lb |
Boolean Updatable |
Use NSX-T layer 4 virtual server for each Kubernetes service of type LoadBalancer. Values: true , false .Default: true . |
x_forwarded_for |
String Updatable |
Sets the original client source IP in the request header. Enabling the network profile x_forwarded_for parameter automatically enables the x_forwarded_port and x_forward_protocol parameters.Values: “none” , “insert” , “replace” .Default: “none” . |
max_l4_lb_service |
Integer Updatable |
Limit the maximum number of layer 4 virtual servers per cluster. Minimum Value: 1 .See also: l4_lb_algorithm and l4_persistence_type . |
l4_persistence_type |
String Updatable |
Connection stickiness based on source_ip .Values: “source_ip” .See also: l4_lb_algorithm and max_l4_lb_service . |
l4_lb_algorithm |
String Updatable |
Layer 4 load balancer behavior. Values: “round_robin” , “least_connection” ,“ip_hash” , “weighted_round_robin” .Default: “round_robin” . See also: l4_persistence_type and max_l4_lb_service . |
The nsx_lb
parameter is used to control the TCP layer 4 virtual server that is provisioned for each Kubernetes service of type: LoadBalancer.
When you configure an NSX Load Balancer as your Kubernetes cluster ingress resource, NCP instructs the NSX-T Load Balancer to provision two layer 7 virtual services (HTTP and HTTPS) as the cluster Ingress Controller:
nsx_lb setting | Description |
---|---|
nsx_lb: true |
Use an NSX-T Layer 4 LoadBalancer and NCP-provisioned Layer 7 Ingress Controller. |
nsx_lb: false |
Use a third-party load balancer and a third-party ingress controller, such as NGINX. |
The nsx_lb
flag controls whether to deploy either the NSX-T Load Balancer or a third-party load balancer, such as Nginx.
The nsx_lb
parameter accepts true
or false
. The default setting is true
and the NSX-T Load Balancer is deployed. To use a third party load balancer, set this parameter to false
.
For example:
{
"name": "example_network_profile",
"description": "nsx_lb is enabled",
"parameters": {
"cni_configurations": {
"type": "nsxt",
"parameters": {
"nsx_lb": false
}
}
}
}
Note: The nsx_lb
parameter maps to the use_native_loadbalancer
parameter in NCP.ini.
For more information about configuring the NSX Ingress Controller component of the NSX-T Load Balancer, see Defining Network Profiles for the HTTP/S Layer 7 Ingress Controller.
The X-Forwarded-For HTTP header field is used to identify the originating IP address of a client connecting to a web server through an HTTP proxy or load balancer.
In network profile, the x_forwarded_for
parameter can be enabled to ensure that the client IP address will be set in the HTTP request header. The x_forwarded_for
parameter is useful in situations where it is important to know the source IP address of the client request, such as for auditing purposes.
Note: Enabling the network profile x_forwarded_for
parameter automatically enables the x_forwarded_port
and x_forward_protocol
parameters. This is supported with NSX-T v3.0.x and later.
The x_forwarded_for
parameter type is String that accepts "none"
, "insert"
, and "replace"
. Any other type will be rejected. Missing entry is accepted.
If x_forwarded_for
is set to “insert”, the client source IP will be appended (comma separated) to the existing set of client source IP addresses. If x_forwarded_for
is set to “replace”, any existing client source IP address will be replaced with the current client source IP address.
For example, with the following network profile the source IP address of the client will be appended to the existing set of client source IP addresses:
{
"name": "example-network-profile",
"description": "x_forwarded_for insert",
"parameters" : {
"cni_configurations": {
"type": "nsxt",
"parameters": {
"x_forwarded_for": "insert"
}
}
}
}
For example, with the following network profile the existing source IP address of the client will replace all other client source IP entries:
{
"name": "example-network-profile",
"description": "x_forwarded_for replace",
"parameters" : {
"cni_configurations": {
"type": "nsxt",
"parameters": {
"x_forwarded_for": "replace"
}
}
}
}
The default NSX-T Load Balancer behavior is that auto-scaling is unlimited. This means that the number of layer 4 virtual servers that can be deployed is governed only by the capacity of the Edge Cluster where the load balancer service is deployed. As a result, in theory it is possible for a single Kubernetes cluster to use up all of the layer 4 virtual servers that the Edge Cluster can support.
The max_l4_service
parameter sets the upper limit for the number of virtual servers that can be used by a Kubernetes cluster. You can use this parameter to limit the number of virtual servers that can be created per Kubernetes cluster.
The max_l4_lb_service
data type is an integer. The value must be larger or equal to 1. Missing entry is accepted.
For example, the following network profile uses the max_l4_lb_service
parameter to limit the number of layer 4 virtual servers to 100 per cluster:
{
"name": "example_network_profile",
"description": "max_l4_lb_service",
"parameters" : {
"cni_configurations": {
"type": "nsxt",
"parameters": {
"max_l4_lb_service": 100
}
}
}
}
The l4_persistence_type
is used to set connection stickiness based on source_ip
.
The l4_persistence_type
data type is string. The only accepted value is source_ip
.
{
"name": "example_network_profile",
"description": "l4_persistence_type",
"parameters": {
"cni_configurations": {
"type": "nsxt",
"parameters": {
"l4_persistence_type": "source_ip"
}
}
}
}
The l4_lb_algorithm
is used to set the algorithm type for the layer 4 NSX-T Load Balancer service.
The l4_lb_algorithm
data type is string enumeration that accepts one of the following values:
"round_robin"
(default)"least_connection"
"ip_hash"
"weighted_round_robin"
For example, the following network profile specifies the weighted_round_robin
as the load balancer algorithm:
{
"name": "example_network_profile",
"description": "l4_lb_algorithm",
"parameters": {
"cni_configurations": {
"type": "nsxt",
"parameters": {
"l4_lb_algorithm": "weighted_round_robin"
}
}
}
}