Each host in the physical rack is connected to the two ToR switches with separate 10Gb links. On each host, NIC port 1 is connected to ToR switch 1 and NIC port 2 is connected to ToR switch 2 with Link Aggregation (LAG).
The BMC on each host is connected to the management switch over a 1G connection. This connection is used for OOB management. In a multi-rack system, both ToR switches are further connected to a pair of inter-rack switches in a dual-LAG configuration using 40 G links. The inter-rack switches are an aggregation layer for connecting multiple racks.
Cloud Foundation is designed to be resilient to certain network failures. The datapath between hosts and ToR switches can tolerate a failure of one link between the host and ToR switches. Between the ToR and inter-rack switches, the system can tolerate the failure of a ToR switch and/or inter-rack switch.