Best practices for VPLEX WAN-COM
The following section describes best practices for configuring VPLEX WAN-COM.
General connectivity best practices
Keep in mind that:
- In a VPLEX configuration, intracluster connectivity refers to director-to-director communication in the VPLEX cluster.
- In a Metro configuration, intercluster connectivity refers to communication between the VPLEX clusters. The configuration uses the WAN-COM module on each director.
- Customers must purchase VPLEX Metro with the appropriate WAN-COM module because reconfiguration of the hardware module is not supported after the initial installation.
VPLEX FC-WAN-COM connectivity best practices
Take account of the following general best practices when configuring Cisco MDS replication VSANs and FC Inter-Switch Links for VPLEX Metro with FC replication.
When you deploy VPLEX Metro with internal intersite connectivity, ensure that:
- Internal intersite connections terminate at the converged system Cisco MDS switches. For example, an ISL to a local DWDM multiplexor (MUX) or termination of a directly connected point-to-point dark fiber link.
External intersite connections are completely external to the converged systems. VPLEX WAN-COM replication ports connect directly to external switches and no ISLs exist between the converged systems.
- The FC WAN-COM module supports switched fabric, DWDM, and FCIP protocols.
- The FC WAN-COM module does not support using Cisco Inter-VSAN routing for WAN-COM zoning.
- Use independent FC WAN links for redundancy on converged systems.
- Each VPLEX director has two FC WAN ports. You must connect these to separate fabrics to maximize redundancy and fault tolerance.
- Logically isolate replication traffic from other traffic using dedicated VSANs.
Configuring the Inter-Switch Link
Adhere to the following guidelines when configuring Inter-Switch Link (ISL) for trunking and buffer-to-buffer credits and deploying VPLEX Metro with DWDM and SONET.
Adhere to the following best practices:
- Trunking enables interconnected ports to transmit and receive frames in more than one Cisco VSAN over the same physical link using the enhanced ISL (EISL) frame format.
- When trunking mode is disabled, add ISL interfaces to the Cisco VSAN that is being extended before activating the links to ensure that only the Cisco VSAN is extended between converged systems.
- By default, trunk mode is enabled on all FC interfaces, but it takes effect only when in E-port mode.
- An operational E-port with trunk mode enabled is referred to as a TE port.
- The trunk-allowed Cisco VSANs configured for TE ports are used by the trunking protocol to determine the allowed active Cisco VSANs in which frames can be received or transmitted.
- If configuring ISLs with trunking mode enabled, do not add local Cisco VSANs to the trunk-allowed Cisco VSAN list.
- Enable trunking only when multiple Cisco VSANs must be extended between converged systems.
- On the Cisco MDS 9000 switches, set the primary converged system end of the trunk setting to “on” and the secondary converged system end to “auto.”
For more information about how to configure inter-switch links between converged systems, see the following documentation:
Adhere to the following best practices:
- FC uses buffer-to-buffer credits (BB_Credits) as a mechanism for hardware-based flow control, so it is not necessary to switch hardware to discard frames caused by high congestion.
- Standard FC flow control and BB_Credit value are adequate for most short-haul deployments. Additional buffering and WAN-optimized flow control are often needed for longer distances.
- Determining sufficient BB_Credits before use is crucial because miscalculations might lead to performance degradation due to credit starvation.
- Add 20 percent to the calculated BB_Credit value to account for spikes in traffic.
- Credit starvation occurs when the number of available credits reaches zero, preventing all forms of FC transmissions. This condition triggers a timeout value, causing the ISL link to reinitialize.
- Depending on the distance between ISL end-points and the MDS switch or switching module being used, BB_Credits must be used to ensure optimal operation.
The following table provides guidelines for determining how many BB_Credits are required based on the distance and speed of the ISL. If the calculated value exceeds the default value, adjust the ISL interface configuration.
- Recommended buffer-to-buffer credit configuration settings
ISL link speed (Gbps)
BB credits per km
The following table shows the default and maximum values for the buffer-to-buffer credit configuration settings per ISL.
Note: The Cisco MDS 9148 Multilayer Fabric Switch has a maximum of 128 BB_Credits per port group.
- Buffer-to-buffer credit configuration settings per ISL
BB_Credits buffers per ISL port
Cisco MDS 9148 Multilayer Fabric Switch
48-Port 8-Gbps FC Module
24-Port 8-Gbps FC Channel Module
DWDM and SONET best practices
If using DWDM or SONET connectivity between converged systems, ensure that the two rings have diverse pathing and that latency is measured for both paths. The following behavior is expected:
- VPLEX directors load-balance (implement round-robin) between the two paths so that any large discrepancy in latency causes VPLEX to operate at speeds based on the slower path.
- VPLEX issues call home events if there is a large discrepancy but does not take action.
Cisco MDS Inter-VSAN Routing (IVR)
Before you configure Cisco MDS Inter-VSAN routing (IVR), review the following information about licensing and the Cisco MDS Enterprise Package part numbers.
Take into account that:
- The Cisco MDS Enterprise Package must enable Inter-VSAN (IVR) routing on each converged system FC switch. IVR is a feature of Advanced Traffic-Engineering.
- IVR allows a selective transfer of data traffic between specific initiators and targets on different VSANs, eliminating the need to merge VSANs into a single logical fabric.
- IVR facilitates resource sharing across VSANs without compromising the VSAN benefits of scalability, reliability, availability, or network security.
- IVR works across WANs using FCIP. FCIP is supported by VPLEX Metro.
- The Cisco MDS Enterprise Package enables zone-based Quality of Service (QoS) to complement the standard QoS that is already available, and extended buffer-to-buffer credits to increase the distance for SAN extension.
- In addition to Advanced Traffic-Engineering, the Cisco MDS Enterprise Package enables Enhanced Network Security with the following features:
- Cisco TrustSec FC Link Encryption
- Switch-switch and host-switch authentication with the FC Security Protocol (FCSP)
- Diffie-Hellman Challenge Handshake Authentication Protocol (DH-CHAP)
- Port security that locks mappings of entities to switch ports
- VSAN-based access control
- IP Security (IPsec) for FCIP
- Digital certificates and fabric binding for open systems
Cisco MDS Enterprise Package
The following table shows the part numbers to use when ordering a Cisco MDS Enterprise Package:
- Cisco MDS Enterprise Package part numbers
Cisco MDS Enterprise Package for one Cisco MDS 9700 Series Multilayer Director
Cisco MD Enterprise Package for one Cisco MDS 9500 Series Multilayer Director
Cisco MDS Enterprise Package for one Cisco MDS 9100 Fabric Switch
VPLEX IP WAN-COM connectivity
Adhere to the following best practices.
- Connect the IP WAN ports to the Cisco Nexus 9000. The IP WAN ports are optical 10 Gbps and do not automatically negotiate down to slower speeds. They must connect to 10 Gbps SFPs.
- Provide 10 Gbps connectivity locally where the IP WAN-COM ports attach to the network. Other network segments can run at speeds other than 10 Gbps.
- Assign all IP WAN ports in port-group 0 to the same VLAN in each site.
- Assign all IP WAN ports in port-group 1 to the same VLAN in each site.
The IP WAN ports do not support 802.1Q tagging.
- Use different VLANs for port-group 0 and port-group 1.
- Configure the switch interfaces as access ports.
- The Maximum Transmission Unit (MTU) size attribute affects IP WAN-COM performance. The default IPv4 MTU size for network switches is 1,500.
- Increasing the size of the MTU increases performance over the WAN.
- VPLEX supports a maximum MTU size of 9,000. Use the highest MTU size that is supported on the network.
- Configure every network switch in the path between the VPLEX clusters to support jumbo frames. Otherwise, the frame is fragmented into multiple smaller frames with an MTU of 1500, which negatively affects performance.
- When jumbo frames are used with IPv6, the routers do not fragment the packet on behalf of the source. Instead, they drop the packet and send back an error message.
- Set the correct socket buffer size according to your anticipated workload. The following values are suggested for starting your base lining process with a specified socket buffer size:
- 1 MB5 MB is optimal for an MTU of 1,500 with an RTT of 1 millisecond.
- 5 MB is optimal for an MTU of 1,500 with an RTT of 10 milliseconds.
- 5 MB is optimal for an MTU of 9,000 with an RTT of 1 millisecond and 10 milliseconds.
For instructions on how to change the MTU size or the socket buffer size, see docu58234 VPLEX IP Networking: Implementation Planning and Best Practices on the Dell Support website.
VPLEX Cluster Witness Server configuration
Before you configure the VPLEX Cluster Witness Server, take into consideration that:
- VPLEX Cluster Witness Server is a VPLEX component that is provisioned as a VM on a VMware ESXi host. VPLEX Witness is typically deployed in a third site or failure domain to enforce isolation from failures that could potentially affect the VPLEX clusters at either site.
- Deploying a VPLEX Metro solution with VPLEX Witness provides continuous availability to the storage volumes if there is a site failure or intercluster link failure (WAN partition).
- VPLEX Witness failure-handling semantics apply only to the distributed vVols in a CG.
- VPLEX Witness server is a mandatory requirement for cross-connect.
- VPLEX Witness server is recommended for non cross-connect.
VPLEX security considerations
Important: A VPLEX Metro system does not support native encryption over an IP WAN-COM link.
Dell Technologies recommends that you deploy an external encryption solution such as IPsec to achieve data confidentiality and end-point authentication over IP WAN-COM links between VPLEX clusters on converged systems.
VPLEX port usage and firewall rules
When configuring VPLEX solutions, apply the recommended guidelines for port usage and firewall rules. Look for the VPLEX Security GeoSynchrony Configuration Guide on the Dell Support website.