Infrastructure nodes are used to host the critical cluster infrastructure services, including:
- NameNode processes
- Ozone Manager
- Ozone Storage Container Manager
- YARN ResourceManager
- ZooKeeper
- HBase masters
- Cloudera Manager
- Supporting databases
Dell Technologies recommends the configuration that is listed in Infrastructure nodes configuration as a starting point. This configuration is optimized for reliability, provides high performance, and is consistent with recommendations from Cloudera.
Machine function | Component |
Platform | PowerEdge R660 server |
Chassis | 1U 2.5 in. chassis with up to 10 hard drives (SAS or SATA), including a maximum of four universal drives, two CPUs, and PERC 11 |
Chassis configuration | Riser configuration 3, full-height, two 16-channel slots (Gen5) |
Power supply | Dual hot-plug power supplies, fully redundant (1+1), 1100 W MM Titanium |
Processor | Dual Intel Xeon Gold 6444Y 3.6 G, 16 C/32 T, 16 GT/s, 45 M cache, turbo, HT (270 W) DDR5-4800 |
Memory | 256 GB - eight 32GB RDIMM, 4800MT/s, dual rank |
Persistent memory | None |
OCP network card | Intel E810-XXV dual port 10/25 GbE SFP28 OCP NIC 3.0 |
Extra network card | NVIDIA ConnectX-6 Lx dual port 10/25 GbE SFP28 PCIe low-profile adapter |
Storage controller | Dell PERC H755 rear SAS RAID controller |
Disk - HDD | None |
Disk - SSD | Dual 480 GB SSD SATA read-intensive, 6 Gbps 512e 2.5 in. hot-plug S4610 drives, 1 Drive Write per Day (DWPD) |
Six 1.6 TB SSD SAS mixed use, up to 24 Gbps 512e 2.5 in. hot-plug AG drives, 3 DWPD | |
Disk - NVMe | Two 1.6 TB enterprise NVMe mixed use U.2, Gen4, P5620 with carrier |
Boot configuration | From PERC controller |
Dell Technologies recommends the disk volume and partition layouts for this set of machines, which are listed in:
Usage | Volume type | Physical disks | Volume Id |
Operating system | RAID 1 | Two 480 GB SATA SSD | 0 |
HDFS metadata and operational databases | RAID 6 | Six 1.6 TB SAS SSD | 1 |
ZooKeeper and NameNode journal | No RAID | One 1.6 TB NVMe | 2 |
Mount point | Size | File system type | Volume Id | Partition type | Description |
/boot | 1024 MB | ext4 | 0 | Primary | This partition contains BIOS start-up files that must be within first 2 GB of disk. |
/ | 100 GB | ext4 | 0 | LVM | This partition contains the root file system. |
swap | 4 GB | swap | 0 | swap | This partition contains the operating system swap space partition. |
/home | 1 GB | ext4 | 0 | LVM | This partition contains the user home directories. |
/var | ~350 GB | ext4 | 0 | LVM | This partition contains variable data like system logging files, databases, mail and printer spool directories, transient, and temporary files. |
/journal/zookeeper | 800 MB | ext4 | 2 | LVM | This partition is used for the ZooKeeper data log. The ZooKeeper configuration property dataLogDir must be changed to match this path at installation time. |
/journal/dfs | 800 MB | ext4 | 2 | LVM | This partition is used to store the NameNode transactions (edits) files. The configuration property dfs.namenode.edits.dir must be changed to match this path at installation time. |
/var/lib/dfs | 2 TB | ext4 | 1 | LVM | This partition is used for the NameNode (fsimage) table. The configuration property dfs.namenode.name.dir must be changed to match this path at installation time. |
/var/lib/zookeeper | 500 MB | ext4 | 1 | LVM | This partition is used for the ZooKeeper database. The ZooKeeper configuration property dataDir must be changed to match this path at installation time. |
/var/lib/pgsql | ~4 TB | ext4 | 1 | LVM | This partition contains the operational data directory for databases. This directory primarily contains the Cloudera Manager databases, since the PostgreSQL data directory (PGDATA) is typically /var/lib/pgsql. Alternatives to PostgreSQL should be configured to store their datafiles here. |
Dell Technologies recommends this Infrastructure node configuration for Master nodes in the cluster. The configuration is sized to support Master nodes in a production deployment.
Gateway nodes and Utility nodes should use this configuration as a starting point. You can change the processor, memory, and storage recommendations to specialize those nodes.
The configuration includes four network ports to provide two ports for the Cluster Data network, and two ports for the Edge network or other external connections.
Two SSDs in a RAID 1 configuration are used for the operating system volume. The swap partition is small since swapping causes excessive latency for critical cluster infrastructure. The home directories are allocated in a separate small partition since user files should not be stored on infrastructure nodes. Most of the storage is allocated to the /var partition for runtime files. You can use LVM to adjust the storage allocation between /, /home, and /var for specific needs.
A six SSD RAID 6 volume is used for most of the Infrastructure node storage. This configuration provides a good balance between performance, data durability, storage efficiency, and administration overhead. This volume is divided into partitions for:
- NameNode file system data
- ZooKeeper data
- Cloudera Manager database storage
- Hive metastore
- Ranger database
- Any other required operational databases
You can use LVM to adjust the storage allocation for specific needs.
An alternative configuration is to set up a four SSD RAID 10 volume for database storage, and a two SSD RAID 1 for NameNode and ZooKeeper data. This configuration option provides slightly better performance for database writes, but adds administration overhead and increases recovery time if drives fail.
A single NVMe drive is used for storage of ZooKeeper and quorum journals.
For clusters which use both Ozone and HDFS storage, the 1.6 TB NVMe drives should be replaced with two 3.2 TB NVMe drives in a RAID 1 configuration. For clusters which only use Ozone, two 6.4 TB NVMe drives should be used.