Home > Storage > PowerMax and VMAX > Storage Admin > Dell PowerMax and VMware vSphere Configuration Guide > Optimizing for interoperability
Dell PowerMax storage arrays offer customers a wide choice of hard drives to meet different workloads including high-performance Storage Class Memory (SCM) and Enterprise Flash Drives (EFD). On the PowerMax, various drive sizes can be intermixed on the same storage array. This feature provides customers the option of providing different applications with the appropriate service level.
The PowerMax storage array is a high-performance NVMe backend delivering increased IOPS and lower latency. NVMe is a high-performance protocol that is designed for modern media. It was architected to take advantage of the parallelism of modern CPUs, SSDs, and SCM to overcome the limitations of storage protocols designed for hard disks. PowerMax platforms exhibit significantly improved data efficiency by using deduplication to reduce the number of duplicate copies of identical data.
A service level is a value or property that is assigned to a storage group on the PowerMax. It defines a target, average response time for I/O operations providing storage performance simplification for users. Each service level has a priority relative to the other service levels. A higher priority service level provides a faster response time than a lower one. Each service level also has a compliance range of response times (since the target is an average) that generate an alert when violated. Some service levels also have a “floor” response time, meaning I/O cannot be serviced quicker than that value. The PowerMax throttles I/O at the front-end to meet this floor to ensure that higher priority service levels are not impacted.
Service levels are addressed more fully in Service levels in Chapter 5.
The most common configuration of a VMware ESXi cluster presents the storage to the virtual machines as flat files in a VMFS. It is tempting to present the storage requirement for the VMware ESXi hosts as one large LUN, as vSphere supports datastores up to 64 TB in size. Using a singular, large LUN can be detrimental to the scalability and performance characteristics of the environment. The next section discusses this issue in detail.
For most environments, Dell using the default queue values for both the HBAs and the VMware specific queues. However, due to performance demands, some customers may require adjustments. VMware may also make such recommendations for I/O intensive workloads or large VMs. While there are no supportability issues using larger queues, Dell Technologies recommends that any queue changes are thoroughly tested before enabling in production. The following discusses HBA and VMware-specific queues, but first a short explanation of queues in an NVMeoF environment is required.
Note: The use of replication on the device does not alter the recommendation.
The PowerMax implementation of NVMeoF supports multiple queues which is a feature of NVMeoF. The array controls the queue setting. There are eight I/O queues per connection or path, each with a queue depth of 128. This arrangement provides excellent scalability. No adjustments are possible.
If environmental conditions require it, VMware has several queues which can be adjusted. Using a few sizeable LUNs in vSphere forces the VMkernel to serially queue I/Os from all the virtual machines using the LUN. To help alleviate the queue, a per-device VMware parameter, Disk.SchedNumReqOutstanding (DSNRO), can be adjusted to prevent one virtual machine from monopolizing the Fibre Channel queue for the LUN. This adjustment is advantageous when more than one virtual machine resides on the device. The HBA queue depth may also require a change to match the new DSNRO value. In most environments, however, the default settings are sufficient and recommended for both the host and VMware queues since the workload is distributed.
Note: Adjusting the DSNRO can have a significant impact on the environment, so caution is warranted. If queuing issues arise that impact performance, a better solution is to add more datastores and redistribute the virtual machines.
VMware also has an adaptive queue depth algorithm that is controlled by two parameters: Disk.QFullSampleSize and Disk.QFullThreshold. Disk.QFullSampleSize is the count of queue full conditions that are required before ESXi starts throttling. It is off by default (zero value). Disk.QFullThreshold is the number of “good” condition responses that are required for ESXi to increase the queue depth limit after reaching the queue full (the value of Disk.QFullSampleSize). It is also off by default. These parameters can be set on a per-device basis. When using PowerMax storage, Dell Technologies recommends leaving the QFullSampleSize and the QFullThreshold parameters unset. The PowerMax arrays would only issue a queue full if the array itself was under duress. In such circumstances, these parameters are unlikely to provide any relief.
A long queue against the LUN results in unnecessary and unpredictable elongation of response time. This problem can be further exacerbated in configurations that allow multiple VMware ESXi hosts to share a single LUN, particularly across VMware clusters. In this configuration, all VMware ESXi hosts using the LUN share the queues that the Fibre Channel port provides. In a large environment with multiple active virtual machines, it is possible to generate a queue that could impact performance. This situation can increase the possibility that VMware and HBA queues require adjusting. This situation can occur despite the ability of PowerMax to share front-end CPU cores and the use of multiple queues on the array.
The potential response time elongation and performance degradation can be addressed by presenting more devices to the VMware ESXi cluster and decreasing the VM-density of each datastore. However, doing so imposes a minor overhead for managing the virtual infrastructure.
The anticipated I/O activity influences the maximum size of the LUN that can be presented to the VMware ESXi hosts. The appropriate size for an environment should be determined after a thorough analysis of the performance data that is collected from existing physical or virtual environments.
Note: When it comes to I/O tuning, Dell Technologies recommends leaving the VMware configuration option DiskMaxIOSize at the default setting of 32 MB.
Note: If multiple storage arrays are presented to the same ESXi hosts along with PowerMax, the VMware queues can be adjusted on a per-device basis to eliminate conflict. This situation is true if the arrays are from Dell (such as PowerStore) or others. It does not hold true for any HBA queues which are host-wide or other VMware-specific parameters. In such cases careful consideration must be taken when adjusting the default values as some arrays can handle higher queues than others. While performance may be improved on one, it could be reduced on another simultaneously. Dell offers recommended settings when using two different Dell storage arrays with the same ESXi host which can be found in Dell KB 303782.
VMFS supports concatenation of multiple SCSI disks to create a single file system. In the past, spanned VMFS could be useful when VMware restricted datastores to 2 TB extents. This situation is no longer a concern as VMware supports 64 TB datastores with a single extent. Dell Technologies does not recommend spanning VMFS as it adds unnecessary complexity to environments and negates the benefits of using PowerMax features particularly around replication. If the intent is to increase queue scalability, and thus performance, multiple datastores should be employed instead.
Virtualization enables better utilization of IT assets. Fortunately, the fundamentals for managing information in the virtualized environment are no different from a physical environment. Dell Technologies recommends the following best practices for a virtualized infrastructure:
Dell makes no general recommendation as to the number or size of datastores (VMFS) in a customer environment, nor how many VMs should be in each datastore. There are many variables which determine the best configuration for a customer and the VM density they can support in each datastore. These variables may include response time, VM size, customer SLAs, type of application, backup and restore requirements, and so forth. To meet these needs, some customers use a few large datastores, for example 8 TB or more. Others prefer a tiered model of small, medium, and large datastores. In the end, if customer performance, availability, and management requirements are met, the configuration is deemed appropriate by Dell.