Customize Node Networks

This topic describes how to define network profiles for customizing Kubernetes node networks provisioned with VMware Tanzu Kubernetes Grid Integrated Edition on vSphere with NSX-T.

Configurable Node Network IP Blocks

The Nodes IP Block is used by Tanzu Kubernetes Grid Integrated Edition to assign address space to Kubernetes nodes when new clusters are deployed or a cluster increases its scale. By default each Kubernetes cluster deployed by Tanzu Kubernetes Grid Integrated Edition is allocated a /24 subnet, which allows up to 256 IP addresses to be assigned.

Using a network profile you can define one or more custom Node IP Block networks, specify the size of the nodes subnet, and specify if the network is routable.

Using the node_ip_block_ids parameter in a network profile, you can specify one or more Nodes IP Blocks for the Kubernetes node networks such that if one of IP block is exhausted, an alternative IP block can be used by Kubernetes clusters to create the Nodes subnet.

Note: Specifying a new node subnet for an existing cluster is not supported. In other words, you cannot autoscale the node network for an existing cluster. For any new clusters created using a network profile with node_ip_block_ids configured, Tanzu Kubernetes Grid Integrated Edition automatically creates a node subnet from one of the IP blocks that is available.

The node_routable boolean lets you specify if the Node network is routable or non-routable. This is the equivalent of enabling or disabling NAT mode in the TKGI tile. If "node_routable":false, the Node network uses NAT mode. In this case you must make sure that Kubernetes nodes have access to BOSH and other TKGI Management Plane components. See Create Management Plane in Installing and Configuring NSX-T Data Center v3.0 for TKGI for more information. If "node_routable":true, the IP address space must be an externally routable address block.

Note: The default routable setting for the Node network is determined based on the selection made in the TKGI tile. If NAT mode is selected, the Node network is non-routable. To override the default selection, provide the node_routable parameter in the network profile.

Depending on the size of the cluster (number of Kubernetes nodes), you can specify a subnet size using the node_subnet_prefix parameter that optimizes the use of network address space. This configuration is especially useful when the cluster nodes are using globally routable address space with the node_routable option set to “true”.

For example, if the Tanzu Kubernetes Grid Integrated Edition administrator has configured the default in the TKGI tile to be a Routable network for the Nodes IP Block*, the Kubernetes cluster administrator can deploy Kubernetes cluster in the NAT’ed mode (non-routable) by specifying a network profile with an IP block that supports the NAT’ed address range.

Note: The default size of the Node network is /24. If you want to use a different size, you must specify the node_subnet_prefix size.

nodes-network.json
{
	"description": "Configurable Nodes Network IP Block",
	"name": "network-profile_nodes-ip-block",
	"parameters": {
		"node_ip_block_ids": [
			"2250dc43-63c8-4bb8-b8cf-c6e12ccfb7de", "3d577e5c-dcaf-4921-9458-d12b0e1318e6"
		],
		"node_routable":true,
		"node_subnet_prefix":20
	}
}