This topic discusses:

• High-level node architecture
• Role assignment recommendations
• Cluster logical networks
• Cluster sizing and scaling
• Cluster licensing
• High availability

High-level node architecture

Figure 8 depicts the Cloudera Data Platform  Data Center high-level architecture.

Figure 8: Node architecture

The cluster environment consists of multiple software services running on multiple physical server nodes.

The implementation divides the server nodes into several roles, and each node has a configuration that is optimized for its role in the cluster. The physical server configurations are divided into two broad classes:

A high-performance network fabric connects the cluster nodes together, and separates the core Data network from management functions.

The minimum supported configuration is eight cluster nodes, which include three Master Nodes, one Utility Node, one Edge Node, and three Worker Nodes. Starting with a ten-node cluster with five Worker Nodes is a common practice. The nodes have the roles that are described in Table 5.

Note: All these nodes roles are required.

Table 5:  Cluster node roles

Table 6 defines the various nodes.

Table 6: Node definitions

Role assignment recommendations

Table 7 describes recommended host role assignments for a medium-sized high availability deployment.

Table 7: CDP Data Center nodes and roles

These recommendations for role assignments are intended as a starting point. Depending on the cluster size and the services that are used, the role assignments may be different. See Runtime Cluster Hosts and Role Assignments in the CDP Data Center documentation for more details.

Cluster logical networks

Three distinct networks are used in the cluster.

Table 8 describes the networks and their purposes.

Table 8: Cloudera Distribution for Apache Hadoop network definitions

Connectivity between the cluster and existing network infrastructure can be adapted to specific installations. Common scenarios are:

1. The Cluster Data network is isolated from any existing network, and access to the cluster is by the Edge network only.
2. The Cluster Data network is exposed to an existing network. In this scenario, the Edge network is either not used or is used for application access or ingest processing.
3. In all cases, the corporate router is used to control access to cluster nodes.

Figure 9  displays cluster logical networks details.

Figure 9: Cluster logical networks

Cluster sizing and scaling

The architecture is organized into three units for sizing as the Hadoop environment grows. From smallest to largest, they are:

• Rack
• Pod
• Cluster

Each has specific characteristics and sizing considerations that are documented in this architecture guide. The design goal for the Hadoop environment is to enable you to scale the environment by adding additional capacity as needed, without replacing any existing components.

Rack sizing

A rack is the smallest size designation for a Hadoop environment. A rack consists of the power, network cabling, and data and management switches to support a group of Worker Nodes.

A rack is a physical unit, and its capacity is defined by physical constraints including available space, power, cooling, and floor loading. A rack should use its own power within the data center, independent from other racks, and should be treated as a fault zone. In the event of a rack level failure in a multiple-rack pod or cluster, the cluster will continue to function with reduced capacity.

This architecture uses 12 nodes as the nominal size of a rack, but higher or lower densities are possible. Typically, a rack will contain about 12 nodes using scale-out servers like the Dell EMC PowerEdge R640 and Dell EMC PowerEdge R740xd. The node density of a rack does not affect overall cluster scaling and sizing, but it does affect fault zones in the cluster.

Pod sizing

A pod is the set of nodes that are connected to the first level of network switches in the cluster, and consists of one or more racks. A pod can include a smaller number of nodes initially, and expand up to these maximums over time.

A pod is a second-level fault zone above the rack level. If a pod level failure occurs in a multiple pod cluster, the cluster continues to function with reduced capacity. A pod can support enough Hadoop server nodes and network switches for a minimum commercial scale installation.

In this architecture, a pod supports up to 36 nodes (nominally three racks). This size results in a bandwidth oversubscription of 2.25:1 between pods in a full cluster. The size of a pod can vary from this baseline recommendation. Changing the pod size affects the bandwidth oversubscription at the pod level, the size of the fault zones, and the maximum cluster size.

Cluster sizing

A cluster is a single Hadoop environment that is attached to a pair of network switches providing an aggregation layer for the entire cluster.

A cluster can range in size from a single pod in a single rack to many pods in multiple racks. A single pod cluster is a special case and can function without an aggregation layer. This scenario is typical for smaller clusters before additional pods are added.

In this architecture, the maximum number of nodes in a cluster depends on the choice of Layer 2 or Layer 3 switching, and the switch models used. See Cluster node counts for the limits.

Cluster node counts

The minimum supported configuration is eight nodes:

• Three Master Nodes
• One Utility Node
• One Edge Node
• Three Worker Nodes

Although a minimum of one Edge Node is required per cluster, larger clusters and clusters with high ingest volumes or rates may require additional Edge Nodes. Cloudera recommends a baseline of one Edge Node for every 20 Worker Nodes.

Table 9 shows the recommended number of nodes per pod and pods per cluster. It also shows some alternatives for cluster sizing with different bandwidth oversubscription ratios.

Note: The network design in this guide uses a 2.25:1 oversubscription ratio.

Table 9: Recommended cluster sizing

Cluster storage sizing

Total cluster storage capacity is a function of the server platform and disk drives chosen, and scales with the number of Worker Nodes.

The amount of usable storage in a cluster also depends on the type of data durability that is used and the type of data compression used. The usable storage capacity can be calculated as:

This calculation is straightforward but depends on estimating the storage efficiency and compression ratio.

The Hadoop Distributed File System (HDFS) storage system supports two options for data durability: replication and erasure coding, and these options have different storage efficiencies.

When replication is used, HDFS creates multiple copies of data across nodes to guard against data loss. The number of replicas (replication factor) is configurable and can be changed on a file by file basis. The default replication factor is three, the value that is normally used for storage capacity estimates.

HDFS replication decreases the storage efficiency by the replication factor.

When erasure coding is used, data is divided into blocks, encoded with parity, and distributed across nodes. The details of the encoding are specified in an erasure coding policy. Erasure coding policies allow a tradeoff between data durability and storage efficiency. For example, a Reed-Solomon 6-3 policy has durability of 3 and a 67% storage efficiency, while a Reed-Solomon 3-2 policy has durability of 2 and a 60% storage efficiency.

Compression can also be used to reduce the storage required. Compression is optional and applies to individual files. HDFS supports multiple data compression codecs, and it is possible for each compressed file to use a different codec. The compression ratio that is achieved for a given file depends on both the dataset and codec that is used and is difficult to estimate. The best approach is to:

1. Test several different codes on real datasets to determine what works best.
2. Make estimates on the amount of data that will be compressed.

The hardware configurations for the Master Nodes support clusters in the petabyte storage range without changes.

Cluster licensing

CDP Data Center licensing for a cluster is based on:

• The number of nodes in a cluster
• The available compute capacity, which is measured in Cloudera Compute Units (CCU)
• The available storage capacity, which is measured in TB

The calculation uses a base price per node, including 16 CCUs and 48 TB. Storage and CCUs are aggregated across the cluster, and variable pricing applies to CCUs and storage above the base allocation for the cluster. As of the publication date of this document, one CCU equals one physical CPU core and 8 GB of RAM.

The recommended configurations in this architecture guide take licensing costs into consideration as part of their designs. See the Cloudera document, Platform pricing, for more details.

High availability

The architecture implements high availability (HA) at multiple levels through a combination of hardware redundancy and software support.