All WLDs can be stretched across two availability zones. Availability zones can be located in either the same data center but in different racks or server rooms, or in two different data centers in two different geographic locations. The following general requirements apply to a VMware Cloud Foundation on VxRail stretched-cluster deployments.

• Witness is deployed at a third site using the same vSphere version that is used in the VCF on VxRail release.
• All SC configurations must be balanced with the same number of hosts in AZ1 and AZ2.

Note: The VI WLD clusters can only be stretched if the Mgmt WLD cluster is first stretched.

The following network requirements apply for the Mgmt WLD and the VI WLD clusters that need to be stretched across the AZs:

• Stretched Layer 2 for the external management traffic.
• Stretched Layer 2 for Edge TEP and Uplink networks for Edge nodes
• 5 millisecond RTT between data node sites
• Layer 3 between each data nodes site and Witness site
• 200 millisecond RTT between data node sites and the Witness site

Figure 43.          Stretched Cluster Network Requirements

The following section contains more detail about the requirements for the network requirements between sites for each type of WLD.

The following table shows the supported connectivity for the data nodes sites for the different traffic types for the Mgmt WLD.

Table 12.           Mgmt WLD Multi-AZ Connectivity Requirements

Management WLD Multi-AZ Connectivity Requirements

The vSAN traffic can only be extended using Layer 3 routed networks between sites. The vMotion traffic can be stretched Layer 2 or extended using Layer 3 routed networks, Layer 3 is recommended. The external management traffic must be stretched Layer 2 only, this ensures the management VMs do not need re-IP when they are restarted on AZ2 if there is a failure of AZ1. The Geneve overlay network can either use the same or different VLANs for each AZ so a single Geneve VLAN but the traffic between sites should be L3 routed. The same VLAN can be used at each site non-stretched, or a different VLAN can be used at each site allowing the traffic to route between sites.

Increasing the vSAN traffic MTU to improve performance requires the MTU for the witness traffic to the witness site to also use an MTU of 9000. This might cause an issue if the routed traffic needs to pass through firewalls or use VPNs for site-to-site connectivity. Witness traffic separation is one option to work around this issue, but is not yet officially supported for VCF on VxRail.

Note: Witness Traffic Separation (WTS) is not officially supported but if there is a requirement to use WTS, the configuration can be supported through the RPQ process. The VCF automation cannot be used for the stretched cluster configuration. The Stretched Cluster with Witness Traffic Separation procedure is a manual VxRail standard procedure that is located in SolVe. To configure WTS, contact your Technical Support representative for guidance.

The requirements for the VI WLD are the same as the Mgmt WLD, if Edge nodes are deployed the edge TEP and uplink networks must be stretched Layer 2 between sites.

Table 13.           VI WLD Multi-AZ Connectivity requirements

During the stretched-cluster configuration, the management VMs are configured to run on the first AZ by default. This is achieved using Host/VM groups and affinity rules that keep these VMs running on the hosts in AZ1 during normal operation. The following diagram shows where the management and NSX VMs are placed after the stretched configuration is complete for the Mgmt WLD and the first cluster of an NSX-T VI WLD.

Figure 44.          Multi-AZ Component Layout

As previously mentioned with AVN enabled, the Edge nodes are deployed and configured to enable the management components in the vRealize Suite to use this network. In the case of multi-AZ, the North/South routing that occurs through AZ1 would need to failover to AZ2 if there is a full site failure. This is achieved by adding the AZ2 TOR switches as BGP neighbors to the Tier 0 gateway so that traffic from the Tier1 can now flow through the TORs at either site. Using both BGP local preference and Path prepend configured on the Tier 0 gateway to steer the traffic out of AZ1 in normal operating conditions requires manual Day 2 configuration. This configuration is outlined in the VVD documentation NSX-T Data Center Configuration for Availability Zone 2 for the Management Domain in Region A.

Figure 45.          Multi AZ – Mgmt WLD VVD routing design