While the networking requirements for VxRail and Cloud Foundation differ, there is overlap in the sense that Cloud Foundation domains depend on the networking resources enabled by VxRail for connectivity. Therefore, the supporting physical network must be properly designed and configured to support VxRail cluster network traffic, and the additional requirements for Cloud Foundation.
Figure 38. VxRail and NSX-T Overlay Networks
A leaf switch is at the lowest tier in a multi-tier architecture, and often referred to as a ‘top-of-rack’ switch. The VxRail nodes will only connect with a leaf switches in a single rack, with the upper tier switches, known as spine switches, enable multi-rack interconnectivity.
The number of Ethernet ports from each VxRail node you reserve for Cloud Foundation on VxRail networking will drive the configuration process for each switch port connected to a VxRail node port. Starting with Cloud Foundation on VxRail version 4.0.1, up to six ports on each node can be reserved for Cloud Foundation on VxRail networking.
- In a 2-port configuration, VxRail network traffic and Cloud Foundation network traffic flow through the same pair of Ethernet ports
- In a 4-port configuration, the VxRail network traffic flows through the first two ports on the network daughter card (NDC). The Cloud Foundation traffic flows either through the other two ports on the NDC, or on two ports on the PCIe expansion card.
- In a 6-port configuration, the VxRail network traffic flows through the four ports on the NDC, and the Cloud Foundation traffic flows through the two ports on the PCIe expansion card.
If the VxRail network traffic and Cloud Foundation network traffic will be physically separated between the nodes and the leaf switches, the VLANs for VxRail and Cloud Foundation only need to be assigned only to the required switch ports.
The following tasks must be performed in the top-of-rack switches in order to prepare for a VxRail cluster deployment and to prepare to support NSX-T:
- Select switches with sufficient open ports capacity to connect all the VxRail nodes, connect the inter-switch links between the leaf switches, and connect upstream to the adjacent network layer.
- Configure at least 1600 MTU to support host overlay network traffic (9000 preferred). A minimum MTU size of 1600 (9000 preferred) must be configured on the leaf switches.
- Ensure that the port type on the switches (RJ45, SFP+) match the port type on the VxRail nodes.
- Configure each of the VLANs required for VxRail on the switches.
- Configure the switch ports to be directly connected to the VxRail nodes as Layer 2 trunk ports.
- Configure unicast on the VLAN representing the vSAN network.
- If opting for VxRail automatic device discovery, configure IPv6 multicast on the VLAN representing the VxRail Internal Management network.
- Configure MLD snooping and MLD querier on the VLAN representing the VxRail Internal Management Network (recommended).
- Configure Spanning Tree on the switch ports to be directly connected to the VxRail nodes as edge ports, or in ‘portfast’ mode.
- If the Layer 2 networks do not terminate at the top-of-rack switches, configure the inter-switch links to allow passage for all VLANs.
- Configure VLAN for the NSX-T host overlay network on each switch.
- If a switch port supports both VxRail network traffic and Cloud Foundation/NSX network traffic, configure all the VLANs for VxRail and Cloud Foundation on the switch ports.
- If a switch port supports only VxRail network traffic or only Cloud Foundation/NSX network traffic, configure only the necessary VLANs on the switch ports.
Each VxRail node has a separate Ethernet port for out-of-band server management called ‘Integrated Dell Remote Access Controller’ (iDRAC). A separate Ethernet switch is recommended to provide connectivity for server maintenance. The server maintenance traffic can also be redirected through the existing network infrastructure. For complete details about VxRail cluster network requirements, see the Dell EMC VxRail Network Planning Guide.
The table in Appendix C: Cloud Foundation on VxRail VLANs lists the individual VLANs that must be configured on the top-of-rack switches. The example switch configuration syntax displayed in Appendix H: Example switch configuration settings offers guidance on how to configure an Ethernet switch with sample VLANs and a sample switch port configuration.