The NSX Advanced Load Balancer can manage load balancing capacity for a virtual service by dynamically scaling it out or in on additional or fewer SEs. By default, the primary SE for the virtual service co-ordinates distribution of traffic flow amongst the secondary SEs, including itself. On OpenStack with Contrail, the NSX Advanced Load Balancer can take advantage of ECMP support offered by Contrail and manage the orchestration of ECMP routes as part of virtual service placement.
ECMP functionality can be homed at the following locations:
The upstream edge router (for instance, Juniper MX or similar ones).
The Contrail vRouter on the host hypervisor.
Example
In Neutron, the VIP 19.1.1.7 is associated with the interface port of the SE named Avi-se-czpey.
root@dc11-cfg-1:~# neutron port-list
+--------------------------------------+------------+-------------------+---------------------------------------------------------------------------------+
| id | name | mac_address | fixed_ips |
+--------------------------------------+------------+-------------------+---------------------------------------------------------------------------------+
| dc4695c6-6d16-4ee0-ab05-774537a33ab2 | Avi-Data...| 02:dc:46:95:c6:6d | {"subnet_id": "a834986a-385e-4616-9d9c-6b91cfaa51e0", "ip_address": "19.1.1.6"} |
| | | | {"subnet_id": "a834986a-385e-4616-9d9c-6b91cfaa51e0", "ip_address": "19.1.1.7"} |
...
+--------------------------------------+------------+-------------------+---------------------------------------------------------------------------------+
In Contrail, the VIP 19.1.1.7 has the same linkage to port dc4695c6-6d16-4ee0-ab05-774537a33ab2 and one route to the SE VM running on hypervisor 172.16.11.101.
Virtual Service Scaleout
Using NSX Advanced Load Balancer CLI, setting scaleout_ecmp to True enables a scaleout of vs2 as follows:
configure virtualservice vs2 scaleout_ecmp save +----------------------------------+-----------------------------------------------------+ | Field | Value | +----------------------------------+-----------------------------------------------------+ | uuid | virtualservice-2f8770d0-20cc-477f-9419-b206b9624389 | | name | vs2 | | scaleout_ecmp | True | | vip[1] | | | vip_id | 0 | | ip_address | 19.1.1.7 | | enabled | True | | network_ref | 02b846e6-cd68-4c84-b7ab-3709946ed28b | | port_uuid | c0e4cb54-8706-4bed-97bb-271a603f858f | | subnet_uuid | a834986a-385e-4616-9d9c-6b91cfaa51e0 | | subnet | 19.1.1.0/24 | | auto_allocate_ip | True | | auto_allocate_floating_ip | False | ... +----------------------------------+-----------------------------------------------------+ scaleout virtualservice vs2 vip_id 0
After the scaleout, vs2 runs on two SEs, Avi-se-czpey and Avi-se-pnjbb, as shown in the following summary output.
show virtualservice vs2 summary +-----------------------+-----------------------------------------+ | Field | Value | +-----------------------+-----------------------------------------+ | oper_status | | | state | OPER_UP | | percent_ses_up | 100 | | vip_summary[1] | | | vip_id | 0 | | oper_status | | | state | OPER_UP | | service_engine[1] | | | ref | Avi-se-czpey | | primary | True | | standby | False | | service_engine[2] | | | ref | Avi-se-pnjbb | | primary | False | | standby | False | | num_se_requested | 2 | | num_se_assigned | 2 | +-----------------------+-----------------------------------------+
In Neutron, the VIP 19.1.1.7 is now associated with the interface ports of SE Avi-se-czpey and Avi-se-pnjbb as well.
root@dc11-cfg-1:~# neutron port-list
+--------------------------------------+------------+-------------------+---------------------------------------------------------------------------------+
| id | name | mac_address | fixed_ips |
+--------------------------------------+------------+-------------------+---------------------------------------------------------------------------------+
| dc4695c6-6d16-4ee0-ab05-774537a33ab2 | Avi-Data...| 02:dc:46:95:c6:6d | {"subnet_id": "a834986a-385e-4616-9d9c-6b91cfaa51e0", "ip_address": "19.1.1.6"} |
| | | | {"subnet_id": "a834986a-385e-4616-9d9c-6b91cfaa51e0", "ip_address": "19.1.1.7"} |
| 492ee72a-c647-490f-9648-77124e1cf0b6 | Avi-Data...| 02:49:2e:e7:2a:c6 | {"subnet_id": "a834986a-385e-4616-9d9c-6b91cfaa51e0", "ip_address": "19.1.1.4"} |
| | | | {"subnet_id": "a834986a-385e-4616-9d9c-6b91cfaa51e0", "ip_address": "19.1.1.7"} |
+--------------------------------------+------------+-------------------+---------------------------------------------------------------------------------+
In Contrail, the VIP 19.1.1.7 has the same linkage to ports dc4695c6-6d16-4ee0-ab05-774537a33ab2 and 492ee72a-c647-490f-9648-77124e1cf0b6, and two routes to the SE VM running on hypervisors 172.16.11.101 and 172.16.11.102.
If there are multiple SE instances for a VIP address, the vRouter will receive multiple XMPP routes towards the final destination. The XMPP routes will have different route distinguishers (RDs) to keep them distinct, and they have different next-hops and MPLS labels to identify the different SE instances.
Flow Resiliency During Scale Out/ In
The flow is 5-tuple: src-IP, src-port, dst-IP, dst-port, and protocol. Routers perform a hash of the 5-tuple to pick a path to use. Contrail vRouter uses flow-tables to preserve the chosen SE destination for a flow. So, even when the number of SE instances in the ECMP group changes, a flow will never move.
Floating-IP ECMP
Using NSX Advanced Load Balancer CLI, associate a floating IP address with virtual service vs2 as follows:
configure virtualservice vs2 vip vip_id 0 auto_allocate_floating_ip Overwriting the previously entered value for auto_allocate_floating_ip floating_subnet_uuid 010a4d99-794c-42a9-948c-080014463217 save save +----------------------------------+-----------------------------------------------------+ | Field | Value | +----------------------------------+-----------------------------------------------------+ | uuid | virtualservice-2f8770d0-20cc-477f-9419-b206b9624389 | | name | vs2 | | scaleout_ecmp | True | | vip[1] | | | vip_id | 0 | | ip_address | 19.1.1.7 | | enabled | True | | network_ref | 02b846e6-cd68-4c84-b7ab-3709946ed28b | | port_uuid | c0e4cb54-8706-4bed-97bb-271a603f858f | | subnet_uuid | a834986a-385e-4616-9d9c-6b91cfaa51e0 | | subnet | 19.1.1.0/24 | | auto_allocate_ip | True | | auto_allocate_floating_ip | False | | floating_ip | 10.1.11.4 | | auto_allocate_floating_ip | True | | floating_subnet_uuid | 010a4d99-794c-42a9-948c-080014463217 | ... +----------------------------------+-----------------------------------------------------+
In Contrail, the floating_ip is linked to the interface ports of both SEs and two routes to the SE VM running on hypervisors 172.16.11.101 and 172.16.11.102.
Virtual Service Traffic Stats
show virtualservice vs2 detail filter disable_aggregate se | grep connections_handled | connections_handled | 102 | | connections_handled | 49 |
The following screenshot shows the ECMP FIP linkage details.
The following screenshot shows the ECMP FIP routing table details.
The following screenshot shows the ECMP linkage details.
The following screenshot shows the non-ECMP linkage details.
The following screenshot shows the Non-ECMP route 1 details.
The following screenshot shows the ECMP route 2 details.
