We configure a 12-node cluster of PowerEdge servers to run SEP. On each of the nodes, SEP is installed on RedHat OpenShift Container Platform. In this section, we provide details of the overall setup along with the installation process and configuration we followed.
The reference architecture cluster comprises of the following components:
- Main cluster: 12 PowerEdge R740xd server nodes:
- Three (3) Red Hat OpenShift Controller nodes:
- These nodes are also used to install one (1) Postgres primary and two (2) replicas.
- Nine (9) Red Hat OpenShift Compute nodes:
- One (1) SEP Coordinator node.
- Seven (7) SEP Worker nodes.
- One (1) node for SEP Hive Metastore and Postgres backup.
- ECS EX500 object storage: Eight (8) nodes.
- Auxiliary cluster: We set up an additional cluster uniquely to test the Stargate functionality and run federated queries across multiple SEP clusters.
Figure 4. Main Cluster Architecture
Figure 5. Multi Custer SEP architecture connected using Stargate
To set up SEP with Dell:
- Set up the Dell PowerEdge R740xd server. For more information, refer to the Dell PowerEdge R740xd Installation and Service Manual.
- Set up networking. The networking configuration and components used in this validation are shown in the illustration below.
Figure 6. Network options
Note: All the nodes use two (2) NIC cards with 2-port 25GbE on each card.
- Set up the Dell ECS EX500. For more information, refer to Dell ECS EXSeries Hardware Guide.
- Deploy the Red Hat Enterprise Linux CoreOS (RHCOS) on the PowerEdge 740xd Servers. For more information, refer to the Red Hat Enterprise Linux CoreOS documentation.
- Deploy the Red Hat OpenShift Container Platform. For more information, refer to the Implementation Guide—Red Hat OpenShift Container Platform 4.10 on Intel-powered Dell Infrastructure.
In this solution we deploy a 3+9 node OpenShift cluster with three Controller nodes and nine Compute nodes. Out of the nine nodes, one is used to deploy Starburst Coordinator, seven are used to deploy Worker nodes and one is used to deploy the Hive Metastore and Postgres backup.
- Deploy SEP on eight Red Hat OpenShift Compute nodes. For more information, refer to the Starburst Enterprise and Kubernetes documentation.
- Install Postgres Primary on one of the Red Hat OpenShift Controller nodes and Postgres Replicas on the remaining 2 Controller nodes. Install Postgres backup on the ninth Compute node along with the Hive Metastore. For more information, refer to the PostgreSQL documentation.
- Install Hive Metastore on the ninth Red Hat OpenShift Compute node. For more information, refer to the Starburst Deploy Hive Metastore with Kubernetes documentation.