To isolate NAS traffic, use a separate VLAN for NFS traffic. Using a separate VLAN also provides an additional layer of security for deployments using NFSv3, which does not support encryption by default. Isilon systems support NFSv4.1, which provides encryption capabilities; however, NFSv4.1 does not support vSphere features such as Storage DRS, Storage I/O Control, and Site Recovery Manager. For this reason, we recommend using NFSv3, which Isilon supports as well.
For optimal performance, enable jumbo frames on the network components servicing Isilon NFS. Testing has shown that enabling jumbo frames delivers 98 to 99 percent efficiency, depending on the packet type, when all hops from the client endpoint to an Isilon node support jumbo frames (see Dell EMC PowerScale: Network Design Considerations). Without jumbo frames enabled, packets likely will be fragmented, leading to additional processing overhead on devices or Path MTU Discovery finding the lowest MTU along the path. Because each workload environment is unique, enable jumbo frames and measure performance enhancements in a lab before updating a production network.
We enabled jumbo frames on the Isilon nodes, physical switches, NICs, and VMkernel ports.
The Mellanox ConnectX-5 supports singe root I/O virtualization (SR-IOV), which allows multiple operating systems to share a physical interconnect. This feature eliminates costly emulation layer overhead between the guest driver and I/O hardware. SR-IOV is enabled by default in the PowerEdge BIOS; however, ensure that it is enabled in ESXi and the
VMs that will use this feature.
The following figure shows the SR-IOV architecture:
Figure 11. Basic SR-IOV architecture
The IEEE 802.3x standard defines an Ethernet flow control mechanism at the Data Link Layer. It specifies a pause flow control mechanism through MAC control frames in full-duplex link segments. For flow control to be successfully implemented, it must be configured for the network hops through which the source and destination endpoints communicate. Otherwise, the pause flow control frames are not recognized and are dropped.
By default, the Isilon OneFS operating system listens for pause frames but does not transmit them, meaning flow control is only applicable when an Isilon node is the source. In the default behavior, the OneFS system recognizes pause frames from the destination. However, pause frames can be enabled for transmission, depending on the NIC.
For sixth-generation Isilon nodes with ix NICs, check for pause frames by running the following command from the OneFS CLI:
infPerf-1# sysctl -d dev.ix.0.mac_stats.xon_txd dev.ix.0.mac_stats.xon_txd: Link XON Transmitted <<<<<<<<
To ensure that NFS network traffic would not oversaturate the network links, we implemented vSphere Network I/O Control (NIOC) and Storage I/O Control (SIOC). Two 25 GbE links per host provide sufficient bandwidth for most general-purpose VMs, however enabling NIOC and SIOC ensures fine-grained control of network traffic such as NFS.
SIOC is used to throttle IOPS to a datastore on a per-VM basis, which prevents “noisy-neighbor” scenarios where certain VMs can monopolize storage bandwidth. NIOC works by setting priority and bandwidth using priority tags in TCP/IP packets. These priority tags allow administrators to control network consumption by traffic type, which is particularly beneficial when using shared network pipes. NIOC and SIOC are complementary traffic-control features.