The 200 GB, 250 GB, and 300 GB per day configurations include:
- Hot and warm bucket storage, that resides on each indexer node, and is sized for 30 days of retention
- Cold bucket storage that is sized for either 120 days or 365 days of retention
Dell EMC Isilon scale-out Network Attached Storage (NAS) can be used for additional cold bucket storage, and for frozen bucket storage.
In order to decide whether to add additional cold bucket storage, or to decouple it from the indexer nodes, consider the following useful points:
- Retention periods for Splunk vary based on ingestion, time periods of search, and high availability.
- The Splunk cold tier provides a searchable data tier. Due to the nature and frequency of data searches in this tier, drive performance is not as critical as for hot and warm tiers.
- Consideration for which option to use depends upon the amount of cold storage you require, and the number of indexers being used.
- Decoupling cold storage using Isilon eliminates the requirement to add additional indexers in order to achieve the proper amount of colder storage.
- If hot and warm retention needs increase, it makes sense to decouple cold storage. Relocating cold drive bays frees up additional space for hot and warm tiers.
For the overall Isilon configuration, Dell EMC also recommends that you:
- Enable SmartPools settings across all Isilon nodes, and use an SSD as L3 cache for random read acceleration.
- Enable SmartConnect to provide automatic client connection load balancing and failover capabilities.
- Enable SmartCache for write performance.
- Use optimization for concurrent data access pattern.
- Use a 40 Gb/s external network for data connection.
- Increase network MTU to 9000 (jumbo frames).
Splunk and Dell EMC recommend that NFS storage, including Isilon storage, is only used for cold and frozen data, never for hot/warm data. For information about system requirements, see the Splunk Enterprise Installation Manual.