Home > Storage > PowerScale (Isilon) > Industry Solutions and Verticals > Media and Entertainment > PowerScale OneFS: NFS over RDMA for Media > What is NFS over RDMA?
Remote Direct Memory Access (RDMA) is a protocol that allows for a client system to copy data from a storage server’s memory directly into that client’s own memory. This allows the client system to bypass many of the buffering layers inherent in TCP. This direct communication improves storage throughput and reduces latency in moving data between server and client. It also reduces CPU load on both the client and storage server.
RDMA was developed in the late 1990s to support high performance compute workloads running over InfiniBand networks. In the 2000s, two methods of running RDMA over Ethernet networks were developed: iWARP and RoCE. There are various benefits and drawbacks in these two approaches:
OneFS’s RDMA implementation uses the RoCE protocol. Or more specifically, OneFS uses the RoCEv2 protocol (also known as “Routable RoCE”’ or “RRoCE”). Given that RDMA in OneFS is meant for maximum storage performance and lowest CPU utilization, the choice of RoCEv2 as the RDMA implementation in OneFS makes sense.
OneFS 9.2+ is required to run NFS traffic over RDMA. NFS over RDMA is enabled in OneFS with a simple checkbox in the global UNIX Sharing settings.
The OneFS hardware must have Mellanox ConnectX-3 Pro, ConnectX-4, or ConnectX-5 front-end NICs. To determine NIC type, run the following commandfrom the OneFS command line:
isi network interfaces list -v
The text “SUPPORTS_RDMA_RROCE” will be present in the flags section on the interfaces that support RDMA.
The client systems must be running a Linux distribution with NIC hardware and drivers that support NFS over RDMA, such as Mellanox ConnectX-3 (and higher) or ATTO FastFrame3 NICs. For details about mounting a OneFS file system using RDMA, see How to Configure NFS over RDMA.
For considerations when preparing an environment for NFS over RDMA traffic, see the section “NFS over RDMA” in Dell EMC PowerScale OneFS Best Practices.
ATTO Technology has also developed OneFS specific profiles that simplify configuring NFS over RDMA with their FastFrame3 family of NICs. For details, see Maximize Ethernet Bandwidth on Dell OneFS Storage with ATTO SmartNICs using NFS over RDMA.