Experience Higher Performance with Splunk® Enterprise
Download PDFTue, 17 Jan 2023 06:53:02 -0000
|Read Time: 0 minutes
Summary
Splunk deployments require unique server and performance characteristics. In this brief document Intel and Dell technologists discuss key considerations to successful Splunk deployments and recommended configurations based on the most recent 15th Generation PowerEdge Server portfolio offerings.
Splunk® Enterprise provides high-performance data analytics for organizations looking for operational, security and business intelligence. With Splunk Enterprise, organizations experience reduced downtime, gain continuous thread remediation and benefit from smarter production insights.
Organizations can experience even higher performance with Splunk Enterprise by selecting the latest Dell EMC™ PowerEdge™ servers. These servers are configured with 3rd Generation Intel® Xeon® Scalable processors and Intel® Ethernet 800 Series network adapters. 3rd Generation Intel® Xeon® Scalable processors deliver an average 46 percent improvement on popular data center workloads, compared to the previous generationi. Intel® Ethernet 800 Series network adapters for OCP3 can help reduce latency and increase application throughput.
Intel and Splunk have partnered to develop recommended configurations for Dell EMC PowerEdge servers. Below, you will find configurations for the Splunk Enterprise admin server, search head and index servers (for either 120-day or 365-day retention) at three performance levels: reference, mid-range and high- performance.
Key Considerations
Splunk users should configure their server infrastructures to match their data-analysis needs. For example, optimizing for low search runtimes requires a different approach than optimizing for high data-ingestion rates.
Before you start, know your use case. Will your Splunk workload ingest data and then index it to make it available for search? Or will your Splunk workload primarily search—that is, query and report? Alternatively, do you envision balancing workloads between ingesting data and searching through data? First characterize your workloads, and then tune your infrastructure as outlined in the following steps:
- Tune your infrastructure for indexing. If most of your workloads ingest and index data, consider increasing the number of parallel ingestion pipelines on the indexer or increasing DRAM capacity.
- Tune your infrastructure for search. If most of your workloads search, consider adding more search heads, adding more computing power on the indexers, or increasing DRAM capacity. For dense search requirements, you might want to turn off hyper-threading.
- Tune balanced workloads. If your workloads are balanced, add indexers when you need to scale.
Recommended Configurations
The recommended configurations for the Splunk Enterprise admin server, search head, and indexers are shown in the table below. Note the following configuration definitions: Reference configuration: Ingestion up to 200 GB per day.
Mid-range configuration: Ingestion up to 250 GB per day.
High-performance configuration: Ingestion up to 300 GB per day.
| Admin Server | Search Head | Indexer (120-day retention) | Indexer (365-day retention) |
Configurations | The admin server and search head have the same configurations for reference, mid- range, and high-performance configurations. | Indexer CPU components are color-coded to indicate configuration. Blue: Reference configuration Green: Mid-range configuration Orange: High-performance configuration | ||
Platform | Dell EMC™ PowerEdge™ R650 supporting 8 x 2.5” Serial- Attached SCSI (SAS)/Serial ATA (SATA) drives | Dell EMC PowerEdge R750 chassis supporting 24 x 2.5” SAS/SATA drives | Dell EMC PowerEdge R750 chassis supporting 24 x + 4 x (rear) 2.5” SAS/SATA drives | |
CPU | 2 x Intel® Xeon® Gold 6326 processor (16 cores at 2.9 GHz) | 2 x Intel® Xeon® Gold 6326 processor (16 cores at 2.9 GHz) | 2 x Intel® Xeon® Gold 6326 processor (16 cores at 2.9 GHz) 2 x Intel® Xeon® Gold 6354 processor (18 cores at 3.0 GHz) 2 x Intel® Xeon® Gold 6348 processor (28 cores at 2.6 GHz) | |
DRAM | 64 GB (8 x 8 GB DDR4- 3200) | 128 GB (8 x 16 GB DDR4-3200) | ||
Boot device | Dell EMC™ Boot Optimized Server Storage (BOSS)-S2 with 2 x 480 GB Intel® SSD S4510 M.2 SATA (RAID1) | |||
Storage adapter | Dell™ PowerEdge RAID Controller (PERC) H345 | Dell PERC H755 | Dell PERC H755 + expander | |
Storage | 2 x 960 GB Intel® SSD S4610 SATA (mixed-use) | 2x 480 GB Intel® SSD S4610 SATA (mixed-use) |
| |
Storage (hot/warm) |
| 6 x 960 GB Intel® SSD S4610 SATA (RAID6) (mixed-use) | ||
Storage (cold tier) | 8 x 2.4 TB 10K rotations per minute (RPM) SAS hard-disk drive (HDD) (RAID6) | 18 x + 4 x (rear) 2.4 TB 10k RPM SAS HDD (RAID6) | ||
Network interface card (NIC) | Intel® Ethernet Network Adapter E810-XXVDA2 for OCP3 (dual-port 25 Gb) |
Learn More
Contact your Dell or Intel account team for a customized quote 1-877-289-3355
Learn more about Dell EMC PowerEdge R750 and R650 servers.
Learn more about high-performance data analytics with Splunk Enterprise running on Intel technologies.
i Source: 125 at Intel. “3rd Generation Intel® Xeon® ® Scalable Processors – Performance Index.” www.intel.com/3gen-Xeon® - config. Results may vary.
Related Documents
Scaling and Optimizing ML in Enterprises
Tue, 16 May 2023 19:53:46 -0000
|Read Time: 0 minutes
Summary
This joint paper, written by Dell Technologies, in collaboration with Intel®, describes the key hardware considerations when configuring a successful MLOps deployment and recommends configurations based on the most recent 15th Generation Dell PowerEdge Server portfolio offerings.
Today’s enterprises are looking to operationalize machine learning to accelerate and scale data science across the organization. This is especially the case as their needs grow to deploy, monitor, and maintain data pipelines and models. Cloud native infrastructure, such as Kubernetes, offers a fast and scalable means to implement Machine Learning Operations (MLOps) by using Kubeflow, an open source platform for developing and deploying Machine Learning (ML) pipelines on Kubernetes.
Dell PowerEdge R650 servers with 3rd Generation Intel® Xeon® Scalable processors deliver a scalable, portable, and cost-effective solution to implement and operationalize machine learning within the Enterprise organization.
Key Considerations
- Portability. A single end-to-end platform to meet the machine learning needs of various use cases, including predictive analytics, inference, and transfer learning.
- Optimized performance. High-performance 3rd Generation Intel® Xeon® Scalable processors optimize performance for machine learning algorithms using AVX-512. Intel® performance optimizations that are built into Dell PowerEdge servers can help fine-tune large Transformers models across multi-node systems. These work in conjunction with open-source cloud native MLOps tools. Optimizations include Intel® and open-source software and hardware technologies such as Kubernetes stack, AVX-512, Horovod for distributed training, and Tensorflow 2.10.0.
- Scalability. As the machine learning workload grows, additional compute capacity needs to be added to the cloud native infrastructure. Dell PowerEdge R750 servers with 3rd Generation Intel® Xeon® Scalable processors deliver an efficient and scalable approach to MLOps.
Recommended Configurations
Cluster | ||
| Control Plane Nodes (Three Nodes Required) | Data Plane Nodes (4 Nodes or More) |
Functions | Kubernetes services | Develop, Deploy, Run Machine Learning (ML) workflows |
Platform | Dell PowerEdge R650 up to 10x 2.5” NVMe Direct Drives | |
CPU | 2x Intel® Xeon® Gold 6326 processor (16 cores @ 2.9GHz), or better | 2x Intel® Xeon® Platinum 8380 processor (40 cores at 2.3 GHz), or 2x Intel® Xeon® Platinum 8368 processor (38 cores @ 2.4GHz), or Intel® Xeon® Platinum 8360Y processor (36 cores @ 2.4GHz) |
DRAM | 128 GB (16x 8 GB DDR4-3200) | 512 GB (16x 32 GB DDR5-4800) |
Boot device | Dell Boot Optimized Server Storage (BOSS)-S2 with 2x 240GB or 2x 480 GB Intel® SSD S4510 M.2 SATA (RAID1) | |
Storage adapter | Not required for all-NVMe configuration. | |
Storage (NVMe) | 1x 1.6TB Enterprise NVMe Mixed- Use AG Drive U.2 Gen4 | 1x 1.6TB (or larger) Enterprise NVMe Mixed-Use AG Drive U.2 Gen4 |
NIC | Intel® E810-XXVDA2 for OCP3 (dual-port 25GbE) | Intel® E810-XXVDA2 for OCP3 (dual-port 25GbE), or Intel® E810-CQDA2 PCIe (dual-port 100Gb) |
Resources
Visit the Dell support page or contact your Dell or Intel account team for a customized quote 1-877-289-3355.
Powering Your Elasticsearch on Kubernetes
Tue, 17 Jan 2023 08:32:07 -0000
|Read Time: 0 minutes
Summary
This joint paper, written by Dell Technologies, in collaboration with Intel®, describes the key hardware considerations when configuring a successful Elasticsearch deployment and recommends configurations based on the most recent 15th Generation PowerEdge Server portfolio offerings.
Elasticsearch is a distributed, open-source search and analytics engine for all types of data, including textual, numerical, geospatial, structured, and unstructured. This proposal contains recommended configurations for Elasticsearch clusters on the Kubernetes platform (Red Hat OpenShift Container Platform with Elastic Cloud on Kubernetes (ECK) operator) running on 15th Generation Dell PowerEdge with 3rd Generation Intel® Xeon® Scalable processors (Ice Lake).
Key Considerations
- Faster and scalable performance. Elasticsearch running on the latest Dell PowerEdge servers is built on high- performing Intel® architecture and configured with 3rd Generation Intel® Xeon® Scalable processors. Indexing is faster and capacity can scale with your needs.
- Index more data. Elasticsearch can handle and store more data by increasing DRAM capacity and using PCIe Gen 4 NVMe disk drives attached to Dell PowerEdge servers.
- Reduced search times and increased # of concurrent searches. As data grows and needs to be accessed across the cluster, data-access response times are critical, especially for real-time analytics applications. Elasticsearch, running on the latest Dell PowerEdge servers, is built on high-performing Intel® architecture. Intel® Ethernet network controllers, adapters, and accessories enable agility in the data center and support high throughput and low latency response times.
- Easy and secure installation. The Elastic Cloud on Kubernetes (ECK) operator is an official Elasticsearch operator certified on Red Hat OpenShift Container Platform, providing easy deployment, management, and operation of Elasticsearch, Kibana, APM Server, Beats, and Enterprise Search on OpenShift clusters. Elasticsearch clusters deployed using this operator are secure by default (with enabled encryption and strong passwords).
- Multi Data Tiers. As data grows, costs do not have to. With multiple tiers of data, capacity can extend, and storage costs can be driven lower without performance loss. Each capacity layer can be scaled independently by using larger drives or mode nodes (or both), depending on customer needs.
Available Configurations
Elasticsearch cluster on Kubernetes (Red Hat OpenShift Kubernetes) platform | ||||
| OpenShift Control Plane Master Nodes (three nodes required) | Elasticsearch Master / Ingest / Hot tier data nodes (minimum of three nodes required) |
Elasticsearch Warm tier data nodes (optional) |
Elasticsearch Cold tier data nodes (optional) |
Functions |
OpenShift services, Kubernetes services | Elasticsearch roles: master, ingest, hot tier data Additional services, such as Kibana |
Elasticsearch roles: warm tier data |
Elasticsearch roles: cold tier data |
Platform |
Dell PowerEdge R650 chassis with up to 10x2.5” NVMe Direct Drives | Dell PowerEdge R750 chassis with up to 12x3.5” HDD with RAID | ||
CPU | 2 x Intel® Xeon® Gold 6326 processor (16 cores @ 2.9GHz) or better |
2 x Intel® Xeon® Gold 6338 processor (32 cores @ 2.0GHz) |
2 x Intel® Xeon® Gold 5318Y processor (24 cores @ 2.1GHz) |
2 x Intel® Xeon® Gold 5318N processor (24 cores @ 2.1GHz) |
DRAM | 128GB (16x 8GB DDR4- 3200) |
256 GB (16 x 16 GB DDR4-3200) | 128 GB (16 x 8 GB DDR4-3200) | |
Boot Device | Dell BOSS-S2 with 2x 240GB or 2x 480GB M.2 SATA SSD (RAID1) | |||
Storage adapter |
Not needed for all-NVMe configurations | Dell PERC H755 SAS/SATA RAID adapter | ||
Storage (NVMe) |
1x 1.6TB Enterprise NVMe Mixed-Use AG Drive U.2 Gen4 |
2x (up to 10x) 3.2TB Enterprise NVMe Mixed-Use AG Drive U.2 Gen4 |
10x 7.68TB Enterprise NVMe Read-Intensive AG Drive U.2 Gen4 |
up to 12x 16TB / 18TB / 20TB 12Gbps SAS ISE 3.5” HDD, 7200RPM |
NIC | Intel E810-XXVDA2 for OCP3 (dual-port 25GbE) |
Note: This document may contain language from third-party content that is not under Dell Technologies’ control and is not consistent with current guidelines for Dell Technologies’ own content. When such third-party content is updated by the relevant third parties, this document will be revised accordingly.
Resources
For more information:
- Contact your Dell or Intel® account team for a customized quote, at 1-877-ASK-DELL (1-877-275-3355).
- See the following documents:
Elastic Cloud on Kubernetes is now a Red Hat OpenShift Certified Operator