Powering Your Elasticsearch on Kubernetes
Download PDFTue, 17 Jan 2023 08:32:07 -0000
|Read Time: 0 minutes
Summary
This joint paper, written by Dell Technologies, in collaboration with Intel®, describes the key hardware considerations when configuring a successful Elasticsearch deployment and recommends configurations based on the most recent 15th Generation PowerEdge Server portfolio offerings.
Elasticsearch is a distributed, open-source search and analytics engine for all types of data, including textual, numerical, geospatial, structured, and unstructured. This proposal contains recommended configurations for Elasticsearch clusters on the Kubernetes platform (Red Hat OpenShift Container Platform with Elastic Cloud on Kubernetes (ECK) operator) running on 15th Generation Dell PowerEdge with 3rd Generation Intel® Xeon® Scalable processors (Ice Lake).
Key Considerations
- Faster and scalable performance. Elasticsearch running on the latest Dell PowerEdge servers is built on high- performing Intel® architecture and configured with 3rd Generation Intel® Xeon® Scalable processors. Indexing is faster and capacity can scale with your needs.
- Index more data. Elasticsearch can handle and store more data by increasing DRAM capacity and using PCIe Gen 4 NVMe disk drives attached to Dell PowerEdge servers.
- Reduced search times and increased # of concurrent searches. As data grows and needs to be accessed across the cluster, data-access response times are critical, especially for real-time analytics applications. Elasticsearch, running on the latest Dell PowerEdge servers, is built on high-performing Intel® architecture. Intel® Ethernet network controllers, adapters, and accessories enable agility in the data center and support high throughput and low latency response times.
- Easy and secure installation. The Elastic Cloud on Kubernetes (ECK) operator is an official Elasticsearch operator certified on Red Hat OpenShift Container Platform, providing easy deployment, management, and operation of Elasticsearch, Kibana, APM Server, Beats, and Enterprise Search on OpenShift clusters. Elasticsearch clusters deployed using this operator are secure by default (with enabled encryption and strong passwords).
- Multi Data Tiers. As data grows, costs do not have to. With multiple tiers of data, capacity can extend, and storage costs can be driven lower without performance loss. Each capacity layer can be scaled independently by using larger drives or mode nodes (or both), depending on customer needs.
Available Configurations
Elasticsearch cluster on Kubernetes (Red Hat OpenShift Kubernetes) platform | ||||
| OpenShift Control Plane Master Nodes (three nodes required) | Elasticsearch Master / Ingest / Hot tier data nodes (minimum of three nodes required) |
Elasticsearch Warm tier data nodes (optional) |
Elasticsearch Cold tier data nodes (optional) |
Functions |
OpenShift services, Kubernetes services | Elasticsearch roles: master, ingest, hot tier data Additional services, such as Kibana |
Elasticsearch roles: warm tier data |
Elasticsearch roles: cold tier data |
Platform |
Dell PowerEdge R650 chassis with up to 10x2.5” NVMe Direct Drives | Dell PowerEdge R750 chassis with up to 12x3.5” HDD with RAID | ||
CPU | 2 x Intel® Xeon® Gold 6326 processor (16 cores @ 2.9GHz) or better |
2 x Intel® Xeon® Gold 6338 processor (32 cores @ 2.0GHz) |
2 x Intel® Xeon® Gold 5318Y processor (24 cores @ 2.1GHz) |
2 x Intel® Xeon® Gold 5318N processor (24 cores @ 2.1GHz) |
DRAM | 128GB (16x 8GB DDR4- 3200) |
256 GB (16 x 16 GB DDR4-3200) | 128 GB (16 x 8 GB DDR4-3200) | |
Boot Device | Dell BOSS-S2 with 2x 240GB or 2x 480GB M.2 SATA SSD (RAID1) | |||
Storage adapter |
Not needed for all-NVMe configurations | Dell PERC H755 SAS/SATA RAID adapter | ||
Storage (NVMe) |
1x 1.6TB Enterprise NVMe Mixed-Use AG Drive U.2 Gen4 |
2x (up to 10x) 3.2TB Enterprise NVMe Mixed-Use AG Drive U.2 Gen4 |
10x 7.68TB Enterprise NVMe Read-Intensive AG Drive U.2 Gen4 |
up to 12x 16TB / 18TB / 20TB 12Gbps SAS ISE 3.5” HDD, 7200RPM |
NIC | Intel E810-XXVDA2 for OCP3 (dual-port 25GbE) |
Note: This document may contain language from third-party content that is not under Dell Technologies’ control and is not consistent with current guidelines for Dell Technologies’ own content. When such third-party content is updated by the relevant third parties, this document will be revised accordingly.
Resources
For more information:
- Contact your Dell or Intel® account team for a customized quote, at 1-877-ASK-DELL (1-877-275-3355).
- See the following documents:
Elastic Cloud on Kubernetes is now a Red Hat OpenShift Certified Operator