While the networking requirements for VxRail and Cloud Foundation differ, there is overlap in the sense that Cloud Foundation domains depend on the networking resources enabled by VxRail for connectivity. Therefore, the supporting physical network must be properly designed and configured to support VxRail cluster network traffic, as well as the additional requirements for Cloud Foundation.

Figure 23            VxRail and NSX-T Overlay Networks

The key points to understand regarding the networking for the VxRail cluster are:

• Each VxRail node requires a minimum of two physical ports to be reserved for Cloud Foundation on VxRail network traffic, and only NSX-V will be used to support the workload domains.
• Each VxRail node requires four physical ports to be reserved if NSX-T is to be deployed for workload domains within Cloud Foundation on VxRail
• Each VxRail node balances the physical connections between the first top-of-rack (ToR) switch and the second top-of-rack (ToR) switch.
• Virtual LANs (VLANs) must be assigned to required VxRail, VXLAN, and AVN (Application Virtual Network) networks to isolate the network traffic, if this option is selected.
• The assigned VLANs must be configured on the Ethernet switches that directly connect to the VxRail nodes.
• The switch ports directly connected to the ports on the VxRail nodes must be trunked ports and allow passage for all of the required network VLANs.
• The inter-switch link between the switches connected directed to the VxRail nodes must also allow passage of all of these VLANs.
• The uplink ports configured for passage upstream must be configured to allow passage for all VLANs requiring external connectivity.
• The AVN (Application Virtual Network) networks, if selected for deployment, must be configured for passage upstream to BGP routing services
• Each VxRail node has a separate Ethernet port for out-of-band server management called ‘Integrated Dell Remote Access Controller’ (iDRAC). A separate Ethernet switch is recommended to provide connectivity for server maintenance. The server maintenance traffic can also be redirected through the existing network infrastructure.

The VLAN for the VXLAN overlay network and the VLANs for the optional Application Virtual Network (AVN) are not VxRail cluster requirements. However, these VLANs are required for Cloud Foundation network operations. The VLANs must be configured on the top-of-rack switches, and also be configured to pass through the trunk ports directly connected to the VxRail nodes.

The following tasks must be performed in the physical top-of-rack switches in order to prepare for a VxRail cluster:

1. Select switches with sufficient open ports for Cloud Foundation on VxRail. If you plan to use NSX-T with any VI workload domains, reserve an additional two Ethernet ports on the switches.
2. Configure at least 1600 MTU to support VXLAN & GENEVE network traffic.
3. Make sure the port type on the switches (RJ45, SFP+) match the port type on the VxRail nodes.
4. Configure each of the VLANs required for VxRail networks on the switches.
5. Configure the switch ports to be directly connected to the VxRail nodes as Layer 2 trunk ports.
6. Configure the inter-switch links to allow passage for all VLANs.
7. Configure the uplinks to allow passage for VLANs requiring external access.
8. Configure unicast on the VLAN representing the VSAN network.
9. Configure multicast on the VLAN representing the VxRail Internal Management network.
10. Configure MLD snooping and MLD querier on the VLAN representing the VxRail Internal Management Network (recommended).
11. Configure Spanning Tree on the switch ports to be directly connected to the VxRail nodes as edge ports, or in ‘portfast’ mode.

For complete details about VxRail cluster network requirements, see the  Dell EMC VxRail Network Planning Guide.