Dell Technologies and Intel engineers deployed two DataStax Enterprise configurations that mirrored a typical, yet smaller-scale, Apache Cassandra deployment. The first configuration used Red Hat OpenShift, while the second configuration used VMware Tanzu. Both of these configurations are the foundation of modern Kubernetes container environments.

Red Hat OpenShift configuration

The Red Hat OpenShift deployment consisted of five Dell EMC PowerEdge R640 servers running Red Hat OpenShift Container Platform 4.6.6. Because of the smaller test environment, there were no separate control-plane nodes. All control-plane and management containers ran on the same servers as Apache Cassandra, which allowed better resource utilization. Larger production clusters that contain additional management and monitoring components would use dedicated control-plane nodes with monitoring and logging tools.

Red Hat OpenShift was installed on Red Hat Enterprise Linux CoreOS (RHCOS), a lightweight container-based Linux operating system, with OpenShift software-defined networking (SDN). The engineers followed the Red Hat OpenShift installation procedures as described in the Red Hat OpenShift documentation.

Two additional components were used with the Red Hat OpenShift configuration:

• Local Storage Operator—This component exposed each server’s local storage devices as persistent volumes that are used by the Apache Cassandra workload.
• Keepalived and External IP—These components exposed Kubernetes services and workloads to the external network, which is necessary in deployments where additional external load balancers are not being used. These components can be installed and configured using the OperatorHub in the OpenShift Container Platform web console. The node configuration can be viewed using the following command:

# kubectl –context ocp get nodes
NAME   STATUS   ROLES                        AGE   VERSION
ocp1   Ready    loadbalancer,master,worker   62d   v1.19.0+43983cd
ocp2   Ready    loadbalancer,master,worker   62d   v1.19.0+43983cd
ocp3   Ready    loadbalancer,master,worker   62d   v1.19.0+43983cd
ocp4   Ready    loadbalancer,worker          62d   v1.19.0+43983cd
ocp5   Ready    loadbalancer,worker          62d   v1.19.0+43983cd

• DataStax Kubernetes Operator for Apache Cassandra—This component was used to ease Apache Cassandra database deployment and life cycle management.

Apache Cassandra is responsible for data redundancy and replication, so there is no need for a storage solution that provides these functions. This capability helps reduce costs and complexity, and it can also scale with the number of Apache Cassandra nodes. Enterprises can use standard Linux Logical Volume Manager (LVM) commands to create smaller logical volumes for each physical server, or they can combine multiple storage devices into a large volume. For this test, each server contained a single volume on each storage device.

Each server was running multiple Apache Cassandra instances to allow better resource utilization and data redundancy. Each server was assigned to a separate logical Apache Cassandra rack, which helps ensure that multiple copies of the same data never reside on the same physical server. With a replication factor of 3 (RF=3), the deployment contained 15 pods running Apache Cassandra database instances. Each pod was assigned 16 logical CPUs and 48 GB of RAM. The pod configuration can be viewed using the following command:

# kubectl –context ocp get pods

NAME                             READY   STATUS    RESTARTS   AGE
cass-operator-6b76b647ff-6sb7h   1/1     Running   0          9d
cluster1-dc1-racka-sts-0         2/2     Running   0          2d20h
cluster1-dc1-racka-sts-1         2/2     Running   0          2d20h
cluster1-dc1-racka-sts-2         2/2     Running   0          2d20h
cluster1-dc1-rackb-sts-0         2/2     Running   0          2d20h
cluster1-dc1-rackb-sts-1         2/2     Running   0          2d20h
cluster1-dc1-rackb-sts-2         2/2     Running   0          2d20h
cluster1-dc1-rackc-sts-0         2/2     Running   0          2d20h
cluster1-dc1-rackc-sts-1         2/2     Running   0          2d20h
cluster1-dc1-rackc-sts-2         2/2     Running   0          2d20h
cluster1-dc1-rackd-sts-0         2/2     Running   0          2d19h
cluster1-dc1-rackd-sts-1         2/2     Running   0          2d19h
cluster1-dc1-rackd-sts-2         2/2     Running   0          2d20h
cluster1-dc1-racke-sts-0         2/2     Running   0          2d19h
cluster1-dc1-racke-sts-1         2/2     Running   0          2d19h
cluster1-dc1-racke-sts-2         2/2     Running   0          2d19h

Red Hat OpenShift server configuration

The storage configuration for each of the five servers varied among the write-intensive, read-intensive, balanced read/write, and high-performance scenarios. The Red Hat OpenShift write-intensive scenario used the Dell EMC PowerEdge R640 server configuration, as shown in the following table:

Table 1.              Dell EMC PowerEdge R640 configuration for Red Hat OpenShift write-intensive scenario

The Red Hat OpenShift read-intensive and balanced read/write scenarios used the Dell EMC PowerEdge R640 server configuration, as shown in the following table:

Table 2.              Dell EMC PowerEdge R640 configuration for the Red Hat OpenShift read-intensive scenario

The Red Hat OpenShift high-performance scenario used the Dell EMC PowerEdge R640 server configuration, as shown in the following table:

Table 3.              Dell EMC PowerEdge R640 configuration for Red Hat OpenShift performance scenario

VMware Tanzu configuration

The VMware Tanzu configuration consisted of five Dell EMC PowerEdge R640 servers running VMware vSphere 7.0 Update 1 with Tanzu. The VMware vCenter Server VM that is used to manage the VMware vSphere cluster was deployed on an additional, separate Dell EMC PowerEdge R640 server. While it is possible to run VMware vCenter on the same VMware vSphere cluster that it manages, VMware recommends running the VMware vCenter VM on a separate management cluster. The engineers followed the installation procedures provided in VMware documentation.

The VMware vSphere cluster was configured with the following:

• Security patches—The latest security patches were applied after the cluster was configured.
• Storage configuration—The five servers were configured with VMware vSAN, which provided high-performance, redundant storage for both VMware vSphere VMs and Tanzu Kubernetes workloads, including Apache Cassandra. The vSAN storage included a cache tier configured using Intel Optane SSDs, in addition to a capacity tier configured using NAND-based SSDs.
• Networking—The VMware vSphere cluster used the vSphere Distributed vSwitch for networking.

When VMware vSphere was fully configured, the engineers installed and configured VMware Tanzu with the following settings:

• Networking—We created port groups and installed the HAProxy virtual application as a load balancer for VMware Tanzu API servers and workload services. This configuration is the only supported load-balancing solution unless the VMware vSphere cluster is using VMware NSX-T Data Center for networking. The Kubernetes worker cluster was deployed using Kubernetes version 1.18.5 and Antrea Container Network Interface (CNI) network. Organizations can also use VMware NSX-T Data Center as a CNI provider.
• Content libraries—The content libraries contained the VMware Tanzu images.
• Supervisor Cluster—We deployed the VMware Tanzu Supervisor Cluster using the tiny deployment size because only a single Kubernetes cluster was running. In a production environment, organizations can install multiple separate Kubernetes clusters on the same physical VMware vSphere infrastructure because VMware Tanzu Kubernetes workers run on their own VMs.
• Kubernetes—The engineers created a single workload namespace with access to all available cluster resources. They deployed a single Kubernetes workload cluster using three control-plane nodes and 10 worker nodes. Because there is no predefined node size, they used the best-effort-8xlarge VM class type for the worker nodes, with each node configured with 32 vCPUs and 128 GB of RAM. The control-plane nodes used the best-effort-xsmall VM class type, with each node configured with 2 vCPUs and 2 GB of RAM. For more information about VMware Tanzu VM class types, see Virtual Machine Class Types for Tanzu Kubernetes Clusters.
• DataStax Kubernetes Operator for Apache Cassandra—This component was used to ease Apache Cassandra database deployment and life cycle management.

When VMware Tanzu is deployed, the node configuration can be viewed using the following command:

# kubectl --context tkg get nodes
NAME                                 STATUS   ROLES    AGE   VERSION
tkg-control-plane-967f5              Ready    master   10d   v1.18.5+vmware.1
tkg-control-plane-n2x96              Ready    master   10d   v1.18.5+vmware.1
tkg-control-plane-pvsnl              Ready    master   10d   v1.18.5+vmware.1
tkg-workers-zqq6s-847b88dd66-4l6s5   Ready    <none>   10d   v1.18.5+vmware.1
tkg-workers-zqq6s-847b88dd66-5kz5w   Ready    <none>   10d   v1.18.5+vmware.1
tkg-workers-zqq6s-847b88dd66-5p4wk   Ready    <none>   10d   v1.18.5+vmware.1
tkg-workers-zqq6s-847b88dd66-6z67r   Ready    <none>   10d   v1.18.5+vmware.1
tkg-workers-zqq6s-847b88dd66-9dhxl   Ready    <none>   10d   v1.18.5+vmware.1
tkg-workers-zqq6s-847b88dd66-fbntb   Ready    <none>   10d   v1.18.5+vmware.1
tkg-workers-zqq6s-847b88dd66-fc5dx   Ready    <none>   10d   v1.18.5+vmware.1
tkg-workers-zqq6s-847b88dd66-g2tmg   Ready    <none>   10d   v1.18.5+vmware.1
tkg-workers-zqq6s-847b88dd66-gqj2n   Ready    <none>   10d   v1.18.5+vmware.1
tkg-workers-zqq6s-847b88dd66-qcwrq   Ready    <none>   10d   v1.18.5+vmware.1

With 10 virtual Kubernetes workers running on five physical server nodes, the engineers created nine Apache Cassandra node instances and grouped them into three logical racks using a replication factor of 3 (RF-3). Each Apache Cassandra pod was running on a separate worker node, with each pod assigned 16 logical CPUs and 64 GB of RAM. The physical servers had Intel Hyper-Threading Technology (Intel HT Technology) enabled, so the 32 CPUs assigned to each worker node were logical cores, not physical cores. Unfortunately, there is some discrepancy in how VMware vSphere and VMware Tanzu count CPU cores with Intel HT Technology enabled, so organizations must consider this discrepancy when planning their own deployments. For more information about this discrepancy, see “vSphere 7 with Kubernetes” in Performance Best Practices for VMware vSphere 7.0.

The pod configuration can be viewed using the following command:

 # kubectl --context tkg get pods

 NAME                             READY   STATUS    RESTARTS   AGE
 cass-operator-557f6689df-527mq   1/1     Running   0          9d
 cluster2-dc1-rack1-sts-0         2/2     Running   0          2d20h
 cluster2-dc1-rack1-sts-1         2/2     Running   0          2d20h
 cluster2-dc1-rack1-sts-2         2/2     Running   0          2d20h
 cluster2-dc1-rack2-sts-0         2/2     Running   0          2d20h
 cluster2-dc1-rack2-sts-1         2/2     Running   0          2d20h
 cluster2-dc1-rack2-sts-2         2/2     Running   0          2d20h
 cluster2-dc1-rack3-sts-0         2/2     Running   0          2d20h
 cluster2-dc1-rack3-sts-1         2/2     Running   0          2d20h
 cluster2-dc1-rack3-sts-2         2/2     Running   0          2d20h

VMware Tanzu server configuration

The storage configuration for each of the five servers varied among the write-intensive, read-intensive, balanced read/write, and high-performance scenarios. The VMware Tanzu write-intensive scenario used the Dell EMC PowerEdge R640 server configuration, as shown in the following table:

Table 4.              Dell EMC PowerEdge R640 server configuration for the VMware Tanzu write-intensive scenario

The VMware Tanzu read-intensive and balanced read/write scenarios used the Dell EMC PowerEdge R640 server configuration, as shown in the following table:

Table 5.              Dell EMC PowerEdge R640 server configuration for the VMware Tanzu read-intensive and balanced read/write scenarios

Because VMware Tanzu does not support Intel Optane PMem, the high-performance scenario was not run on VMware Tanzu.