This section describes how to design and implement the SAP HANA scale-out solution using PowerEdge MX kinetic infrastructure that was tested and validated in Dell EMC laboratories. The SAP HANA TDI scale-out solution described in this guide uses the following PowerEdge MX platform components and Dell EMC enterprise storage:
The SAP HANA scale-out solution requires numerous cables such as FC and Ethernet connections using rack servers in a fully redundant fabric configuration. The following figure shows a logical view of the minimum different network requirements:
Figure 11. SAP HANA scale-out network configuration using MX9116n/MX7116n switches
A significant benefit of using the MX PowerEdge platform for an SAP HANA scale-out solution is its modular network infrastructure. This infrastructure helps to eliminate most network/fabric cabling from SAP HANA nodes to the virtual switch layer. Only switch interconnect links and network/fabric uplinks are required, as shown in the following figure:
Figure 12. Network cabling for an SAP HANA scale-out solution using two PowerEdge MX7000 chassis
The preceding illustration shows two PowerEdge MX7000 chassis, which can support up to eight MX840c servers (with four CPU sockets each), cascaded in a pair of Dell EMC modular network MX9116n/MX7116n switches. For illustration purposes, we used Dell EMC Unity array 650F and other network/fabric uplinks.
The PowerEdge MX7000 chassis is a 7U enclosure with eight front slots for blade-based servers (for up to four PowerEdge MX840c nodes) or storage sleds. The MX7000 chassis supports up to two pairs of redundant general-purpose switches or pass-through modular bays (Fabrics A and B) and a redundant pair of storage-specific switch bays (Fabric C). Up to 25Gbps Ethernet, 32Gbps FC, and 12Gbps SAS are also supported for the server’s front-end I/O ports. The chassis has up to six power supply units (PSUs) as well as OpenManage Enterprise Modular Edition embedded management software running on up to two redundant MX9002m management modules.
The PowerEdge MX platform introduced the concept of scalable fabric architecture. A scalable fabric spans multiple chassis, enabling them to behave like a single chassis from a networking perspective.
A scalable fabric consists of two main components: a pair of MX9116n Fabric Switch Engines (FSEs) in the first two chassis and additional pairs of MX7116n Fabric Expander Modules (FEMs) in the remaining chassis. Each MX7116n FEM connects to the MX9116n FSE corresponding to its fabric and slot. All I/O modules (IOMs) participating in the fabric are configured in either Full Switch or SmartFabric mode.
The Dell EMC Networking MX9116n FSE operates in one of two modes:
The following figure shows three MX7000 chassis in a single scalable fabric architecture. The number of chassis is expandable to ten.
Figure 13. Scalable fabric example using Fabric A
The first two chassis each contain one MX9116n FSE and one MX7116n FEM. As shown in the diagram, chassis 3 to 10 each contain two MX7116n FEMs. All connections shown use QSFP28-DD (2x100GB) connections.
In an SAP HANA scale-out solution using the PowerEdge MX platform, one pair of MX9116n FSEs and additional FEM pairs for additional MX7000 chassis are used for all the required networks, including FC SAN. This helps to consolidate all the required network switches into one scalable fabric, keeping the fabric administration to a minimum and saving the data center footprint compared to an approach using rack servers.
The following figure shows the Dell EMC SAP HANA (16+2) scale-out solution using PowerEdge MX:
Figure 14. Dell EMC SAP HANA (16+2) scale-out solution using PowerEdge MX
The entire hardware requirement can fit in a 42U rack. The previous SAP HANA scale-out solution’s rack design required double the rack space.
The following table shows the hardware components of an SAP HANA scale-out (2+1) solution using PowerEdge MX servers:
Table 13. Dell EMC SAP HANA 6 TB scale-out solution
Component |
Configuration |
SAP HANA nodes |
2 x Dell EMC PowerEdge MX840c active SAP HANA nodes + 1 to 2 MX840c standby SAP HANA nodes. each with:
|
Storage |
|
Storage fabric and network |
One pair of MX9116n modular switches to support both FC and all other Ethernet networks required by the SAP HANA scale-out landscape. |
Two 25 GbE dual port mezzanine cards are used in each of the PowerEdge MX840c servers. 25 GbE ports are partitioned with specific bandwidth allocation, connected to the described fully redundant Scalable Fabric, and configured as bonded network or multipath I/O devices for Ethernet and FC fabric respectively to ensure that there is no single point of failure for the networks that an SAP HANA scale-out landscape requires.
The following figure shows the VLAN ID assignment in the deployed SAP HANA scale-out landscape:
Figure 15. Network port partition on MX840c
The preceding figure and following table show how the 25GE mezzanine ports are partitioned. FC-related NIC partitions are configured as FCoE HBAs for FC SAN fabrics.
A NIC partition such as 2a is allocated to have a minimum transfer bandwidth of 65 percent of the total available 25GE connection. Also, NIC partition 2a can have up to 100 percent of the available transfer bandwidth if NIC partition 1a is idle.
Table 14. CNA configuration within MX840c HANA database hosts
Mezz/CAN slot |
Port number |
Partition number/label |
Partition type |
Partition minimum TX bandwidth |
Partition maximum TX bandwidth |
Application function |
Mezz 1A |
Port 1 |
Partition 1 - 1a |
NIC |
35 |
100 |
NFS network |
Partition 2 - 2a |
FCoE |
65 |
100 |
FC SAN |
||
Port 2 |
Partition 1 - 3a |
NIC |
48 |
100 |
HANA internal network |
|
Partition 2 - 4a |
NIC |
48 |
100 |
HANA external network |
||
Partition 3 - 5a |
NIC |
4 |
100 |
Management network |
||
Mezz 2A |
Port 1 |
Partition 1 - 3a' |
NIC |
48 |
100 |
HANA internal network |
Partition 2 - 4a' |
NIC |
48 |
100 |
HANA external network |
||
Partition 3 - 5a' |
NIC |
4 |
100 |
management network |
||
Port 2 |
Partition 1 - 1a' |
NIC |
35 |
100 |
NFS network |
|
Partition 2 - 2a' |
FCoE |
65 |
100 |
FC SAN |
PowerEdge MX networking switches support Dell EMC OS10 Enterprise Edition and operate in either Full Switch or SmartFabric mode. Users have more control over IOM in Full Switch mode because IOM operates as a regular switch. SmartFabric offers several key features that are crucial for the latest fabric solutions, such as automation, life cycle management, scalability, and manageability.
We implemented our SAP HANA scale-out solution in SmartFabric mode for proof of concept and certification purposes. With SmartFabric Services (SFS), customers can quickly and easily deploy and automate datacenter networking fabrics. The most burdensome tasks were creating NIC partitions from the first SAP HANA server as a template server and creating different VLAN IDs and network uplinks on the MX9116n pair. The template is easily and automatically deployed to the next SAP HANA server as well as its Ethernet and SAN fabric configuration on the switches involved. For more information, see the following documents:
In addition to being able to assign VLANs to server profiles, SFS automates quality of service (QoS) settings based on user input. When a VLAN is created, the user selects the related traffic type—iSCSI, vMotion, and so on—and the SFS engine assigns the correct QoS setting to that VLAN.
Users can also select a “metal” such as gold, bronze, and so on to assign their own priority values to traffic. The following table shows the traffic types and related settings:
Traffic type |
Used for |
QoS setting |
General purpose (bronze) |
Low priority data traffic |
2 |
General purpose (silver) |
Standard/default priority data traffic |
3 |
General purpose (gold) |
High priority data traffic |
4 |
General purpose (platinum) |
Extremely high priority data traffic |
5 |
Cluster interconnect |
Cluster heartbeat VLANs |
5 |
Hypervisor management |
Hypervisor management connections such as the ESXi management VLAN |
5 |
Storage - iscsi |
Used for iSCSI VLANs |
5 |
Storage – fcoe |
Used for FCoE VLANs |
5 |
Storage - data replication |
Used for VLANs supporting storage data replication such as for VMware VSAN |
5 |
VM migration |
Used for VLANs supporting vMotion and similar technologies |
5 |
VMware ft logging |
Used for VLANs supporting VMware fault tolerance |
5 |
Table 15. Traffic types and QoS settings