The core network fabric is based on Ethernet technology. Dell Technologies recommends 25 GbE for new deployments of PowerEdge servers. An aggregation layer is required for clusters larger than a single pod. The aggregation layer can be implemented at either Layer 2 (L2) or Layer 3 (L3). The choice depends on the initial size and planned scaling. Layer 2 is preferred for lower cost and medium scalability, and can support approximately 250 nodes.

Layer 3 aggregation is recommended for:

• Larger initial deployments over 250 nodes
• Instances where the cluster must be co-located with other applications in different racks

The scalability depends on the switches that are used and the oversubscription ratio, and is summarized in Cluster node counts.

Each pod uses a PowerSwitch S5248F-ON as the first layer switch.

The 48-port S5248F-ON provides High-density 10/25 GbE ToR server aggregation with scalable L2 and L3 Ethernet switching and standards-based IPv4 and IPv6 features. It features line-rate performance using nonblocking switch fabrics of 2.0 Tbps (4.0 Tbps full-duplex). It is configured with:

• 48 ports of 25 GbE (SFP28)
• 2 ports of 200 GbE (QSFP28-DD)
• 4 ports of 100 GbE (QSFP28)

25 GbE single pod networking equipment shows the single pod network configuration, with PowerSwitch S5248F-ON switches aggregating the pod traffic.

For a single pod, the ToR switches can act as the aggregation layer for the entire cluster. For multiple-pod clusters, a cluster aggregation layer is required. In this architecture, each pod is managed as a separate entity from a switching perspective. The individual pod ToR switches connect only to the spine switch.

In this design, each pod is managed as a separate entity from a switching perspective, and the individual pod switches connect only to the aggregation switch.

The PowerSwitch Z9432F-ON is used for both Layer 2 and Layer 3 implementations, and can be paired with a second Z9432F-ON switch to create a high availability configuration.

For clusters consisting of more than one pod, this design uses the Z9432F-ON for an aggregation switch. The PowerSwitch Z9432F-ON can be used for both Layer 2 and Layer 3 implementations.

The PowerSwitch Z9432F-ON is a multiple-rate 400 GbE, 1U spine switch optimized for high performance, ultralow latency data center requirements. The PowerSwitch Z9432F-ON can provide a cumulative bandwidth of 25.6 Tbps nonblocking (full duplex). Its switching fabric can deliver line-rate performance under full load. It can be configured with up to:

• 32 ports of 400 GbE (QSFP56-DD)
• 128 ports of 100 GbE (using a breakout cable)
• 144 ports of 10/25/50 GbE (using a breakout cable)

The figure below shows the multiple-pod network configuration with Layer 2 networking.

The uplink from each S5248F-ON pod switch to the aggregation layer uses four 100 GbE interfaces in a bonded configuration. This configuration provides a collective bandwidth of 400 Gb from each pod.

Layer 3 aggregation is used for large clusters and supports scalability beyond a single set of aggregation switches.

Multiple pod view with Layer 3 ECMP illustrates cluster aggregation using the PowerSwitch Z9432F-ON and the PowerSwitch S5248F-ON using a leaf-spine model that is based on ECMP and Layer 3 routing.

More details about Layer 3 leaf-spine deployment can be found in the Dell Technologies Info Hub document, Dell Networking Layer 3 Leaf-Spine Deployment and Best Practices with OS10.

In addition to the Cluster Data network, a separate network is provided for cluster management - the iDRAC or Baseboard Management Controller (BMC) network.

The iDRAC management ports are all aggregated into a per-rack PowerSwitch S3100-ON family switch. This aggregation provides a dedicated iDRAC or BMC network, for hardware provisioning and management. Switch management ports are also connected to this network.

If out-of-band management is required the management switches can be connected to the core, or connected to a dedicated Management network.

The aggregation layer functions as the network core for the cluster. In most instances, the cluster connects to a larger core within the enterprise, as shown in Multiple pod networking equipment.

With the PowerSwitch Z9432F-ON, two 100 GbE ports are reserved for connection to the core. Details of the connection are site-specific and must be determined as part of the deployment planning.

When PowerScale is used for primary HDFS storage, the Cloudera cluster nodes are connected using the same switching architecture that is described above. The PowerScale nodes are connected to their own switching layer and then linked to the aggregation layer.

PowerScale nodes use a dedicated backend network in addition to the Cluster Data network. The backend network is used for internode communication between the PowerScale nodes. This network can use Ethernet or InfiniBand technology.

Every PowerScale node has two 25 GbE connections to the Cluster Data network and two connections to the backend network.

Depending on customer requirements, PowerScale can be networked directly to the leaf topology of the Cloudera CDP Private Cloud Base, or connected to the aggregation layer at the spine. See Cluster sizing and scaling for sizing and scaling information.