Modern data stack storage can use PowerScale with the HDFS protocol, or ECS for object storage. You can use either or both systems depending on workload requirements. Server internal storage provides runtime storage that applications require. The Symcloud Storage software manages runtime storage.

ECS and PowerScale are deployed as cluster-level systems. The node recommendations here can be used as guidance for new clusters, verification of compatibility with existing clusters, or expansion of existing clusters.

Dell Technologies recommends the configuration in PowerScale configuration for storage in clusters using PowerScale for their primary modern data stack storage using HDFS.

The recommended configuration is sized for typical usage as modern data stack HDFS storage.

Two Ethernet network ports per node are included for connection to the Cluster data network or a PowerScale storage network. Two additional network ports are included for connection to the PowerScale back-end network. These additional ports can be either InfiniBand QDR or 40 GbE, depending upon on-site preferences.

One PowerScale H7000 chassis supports four PowerScale H7000 nodes. This configuration provides approximately 720 TB of usable storage. At 85% utilization, 600 TB of HDFS storage is a good guideline for available storage per chassis.

This configuration assumes that the PowerScale nodes are primarily used for HDFS storage. If the PowerScale nodes are used for other storage applications or clusters, you must account for it in the overall cluster sizing. You can also use other PowerScale H7000 drive configurations.

Dell Technologies recommends the configurations in ECS EX500 node configuration or ECS EXF900 node configuration for storage in clusters using ECS for their primary modern data stack storage using the s3a:// protocol.

The ECS EX500 configuration provides a good balance of storage density and performance for modern data stack usage.

Two Ethernet network ports per node are included for connection to the Cluster data network or an ECS storage network. Two additional network ports are included for connection to the ECS back-end network.

The ECS EXF900 configuration is an all-flash configuration and provides the highest performance for modern data stack usage.

Two Ethernet network ports per node are included for connection to the Cluster data network or an ECS storage network. Two additional network ports are included for connection to the ECS back-end network.

The worker nodes include local SSD storage for use as runtime storage for workloads. This storage is exposed to pods running on the cluster through the Kubernetes CSI interface. Persistent and ephemeral volumes are supported. Symcloud Storage manages these storage devices through the Symcloud Storage role.

Symcloud Storage and the associated CSI driver support advanced features including:

• Replication for data availability
• Encryption for data security
• Compression and thin provisioning for storage efficiency
• Snapshots
• Backup and recovery

The Kubernetes storage class defines some of these features, while others are accessible through the Symcloud management interface. More details about the Symcloud Storage capabilities can be found in the Symcloud document, Robin Cloud Native Storage Overview.

The recommended configuration provides approximately 23 TB of storage per node. This capacity is enough for typical runtime storage in a modern data stack environment where the bulk of the data is on external storage. Alternative configurations, including dedicated storage nodes, are possible. Some analytics workloads, like databases, may require additional storage for persistent data outside the modern data stack storage.