This section describes how to design and implement the SAP HANA scale-out solution using PowerEdge MX kinetic infrastructure that was tested and validated in Dell EMC laboratories. The SAP HANA TDI scale-out solution described in this guide uses the following PowerEdge MX platform components and Dell EMC enterprise storage:

• MX7000 chassis—A 7U modular chassis that hosts flexible blocks of resources and provides outstanding efficiency through shared power, cooling, networking, I/O, and management.
• PowerEdge MX networking modules—Modular switches that are designed to reduce network management complexity and grow with business needs.
• PowerEdge MX840c—An SAP HANA certified, four-socket, double-width sled with elastic compute resources for database-driven, mission-critical applications and performance workloads.
• Dell EMC Unity 650F—An SAP HANA certified enterprise storage system with integrated architecture for block file with concurrent support for native NAS, iSCSI, and Fibre Channel (FC)  protocols.

Network requirements

The SAP HANA scale-out solution requires numerous cables such as FC and Ethernet connections using rack servers in a fully redundant fabric configuration. The following figure shows a logical view of the minimum different network requirements:

Figure 11.          SAP HANA scale-out network configuration using MX9116n/MX7116n switches

A significant benefit of using the MX PowerEdge platform for an SAP HANA scale-out solution is its modular network infrastructure. This infrastructure helps to eliminate most network/fabric cabling from SAP HANA nodes to the virtual switch layer. Only switch interconnect links and network/fabric uplinks are required, as shown in the following figure:

Figure 12.          Network cabling for an SAP HANA scale-out solution using two PowerEdge MX7000 chassis

The preceding illustration shows two PowerEdge MX7000 chassis, which can support up to eight MX840c servers (with four CPU sockets each), cascaded in a pair of Dell EMC modular network MX9116n/MX7116n switches. For illustration purposes, we used Dell EMC Unity array 650F and other network/fabric uplinks.

The PowerEdge MX7000 chassis is a 7U enclosure with eight front slots for blade-based servers (for up to four PowerEdge MX840c nodes) or storage sleds. The MX7000 chassis supports up to two pairs of redundant general-purpose switches or pass-through modular bays (Fabrics A and B) and a redundant pair of storage-specific switch bays (Fabric C). Up to 25Gbps Ethernet, 32Gbps FC, and 12Gbps SAS are also supported for the server’s front-end I/O ports. The chassis has up to six power supply units (PSUs) as well as OpenManage Enterprise Modular Edition embedded management software running on up to two redundant MX9002m management modules.

Using Dell EMC modular network modules in PowerEdge MX

The PowerEdge MX platform introduced the concept of scalable fabric architecture. A scalable fabric spans multiple chassis, enabling them to behave like a single chassis from a networking perspective.

A scalable fabric consists of two main components: a pair of MX9116n Fabric Switch Engines (FSEs) in the first two chassis and additional pairs of MX7116n Fabric Expander Modules (FEMs) in the remaining chassis. Each MX7116n FEM connects to the MX9116n FSE corresponding to its fabric and slot. All I/O modules (IOMs) participating in the fabric are configured in either Full Switch or SmartFabric mode.

The Dell EMC Networking MX9116n FSE operates in one of two modes:

• SmartFabric Mode—Switches operate as layer 2 I/O aggregation devices and are managed through the Open Manage Enterprise Modular console.
• Full Switch Mode—This mode is enabled by default, making all switch-specific OS10EE capabilities available.

The following figure shows three MX7000 chassis in a single scalable fabric architecture. The number of chassis is expandable to ten.

Figure 13.          Scalable fabric example using Fabric A

The first two chassis each contain one MX9116n FSE and one MX7116n FEM. As shown in the diagram, chassis 3 to 10 each contain two MX7116n FEMs. All connections shown use QSFP28-DD (2x100GB) connections.

In an SAP HANA scale-out solution using the PowerEdge MX platform, one pair of MX9116n FSEs and additional FEM pairs for additional MX7000 chassis are used for all the required networks, including FC SAN. This helps to consolidate all the required network switches into one scalable fabric, keeping the fabric administration to a minimum and saving the data center footprint compared to an approach using rack servers. 

The following figure shows the Dell EMC SAP HANA (16+2) scale-out solution using PowerEdge MX:

Figure 14.          Dell EMC SAP HANA (16+2) scale-out solution using PowerEdge MX

The entire hardware requirement can fit in a 42U rack. The previous SAP HANA scale-out solution’s rack design required double the rack space.

SAP HANA 6 TB scale-out solution

The following table shows the hardware components of an SAP HANA scale-out (2+1) solution using PowerEdge MX servers:

Table 13.        Dell EMC SAP HANA 6 TB scale-out solution

Two 25 GbE dual port mezzanine cards are used in each of the PowerEdge MX840c servers. 25 GbE ports are partitioned with specific bandwidth allocation, connected to the described fully redundant Scalable Fabric, and configured as bonded network or multipath I/O devices for Ethernet and FC fabric respectively to ensure that there is no single point of failure for the networks that an SAP HANA scale-out landscape requires.

The following figure shows the VLAN ID assignment in the deployed SAP HANA scale-out landscape:

Figure 15.          Network port partition on MX840c

The preceding figure and following table show how the 25GE mezzanine ports are partitioned. FC-related NIC partitions are configured as FCoE HBAs for FC SAN fabrics.

• The Unity array’s FC front-end HBAs are connected through the SAN fabric to MX9116n’s native FC uplinks.
• The MX7000 Fabric B module pair and future support of 25 GbE quad port mezzanine I/O cards can provide more network expansion capability if required.

A NIC partition such as 2a is allocated to have a minimum transfer bandwidth of 65 percent of the total available 25GE connection. Also, NIC partition 2a can have up to 100 percent of the available transfer bandwidth if NIC partition 1a is idle.

Table 14.        CNA configuration within MX840c HANA database hosts

PowerEdge MX networking switches support Dell EMC OS10 Enterprise Edition and operate in either Full Switch or SmartFabric mode. Users have more control over IOM in Full Switch mode because IOM operates as a regular switch. SmartFabric offers several key features that are crucial for the latest fabric solutions, such as automation, life cycle management, scalability, and manageability.

We implemented our SAP HANA scale-out solution in SmartFabric mode for proof of concept and certification purposes. With SmartFabric Services (SFS), customers can quickly and easily deploy and automate datacenter networking fabrics. The most  burdensome tasks were creating NIC partitions from the first SAP HANA server as a template server and creating different VLAN IDs and network uplinks on the MX9116n pair. The template is easily and automatically deployed to the next SAP HANA server as well as its Ethernet and SAN fabric configuration on the switches involved. For more information, see the following documents:

• Dell EMC PowerEdge MX SmartFabric Configuration and Troubleshooting Guide.
• Dell EMC PowerEdge MX Series Fibre Channel Storage Network Deployment with Ethernet IOMs

In addition to being able to assign VLANs to server profiles, SFS automates quality of service (QoS) settings based on user input. When a VLAN is created, the user selects the related traffic type—iSCSI, vMotion, and so on—and the SFS engine assigns the correct QoS setting to that VLAN.

Users can also select a “metal” such as gold, bronze, and so on to assign their own priority values to traffic. The following table shows the traffic types and related settings:

Table 15.        Traffic types and QoS settings